2023-02-08 08:53:54,182 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-08 08:53:54,446 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1 2023-02-08 08:53:54,460 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-08 08:53:54,460 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d, deleteOnExit=true 2023-02-08 08:53:54,460 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-08 08:53:54,461 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/test.cache.data in system properties and HBase conf 2023-02-08 08:53:54,461 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/hadoop.tmp.dir in system properties and HBase conf 2023-02-08 08:53:54,462 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/hadoop.log.dir in system properties and HBase conf 2023-02-08 08:53:54,462 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-08 08:53:54,463 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-08 08:53:54,463 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-08 08:53:54,584 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-08 08:53:54,968 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-08 08:53:54,972 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-08 08:53:54,972 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-08 08:53:54,972 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-08 08:53:54,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 08:53:54,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-08 08:53:54,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-08 08:53:54,973 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 08:53:54,974 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 08:53:54,974 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-08 08:53:54,974 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/nfs.dump.dir in system properties and HBase conf 2023-02-08 08:53:54,974 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/java.io.tmpdir in system properties and HBase conf 2023-02-08 08:53:54,975 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 08:53:54,975 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-08 08:53:54,975 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-08 08:53:55,446 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 08:53:55,451 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 08:53:55,827 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-08 08:53:55,978 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-08 08:53:55,992 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:56,029 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:56,061 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/java.io.tmpdir/Jetty_localhost_41129_hdfs____pkebet/webapp 2023-02-08 08:53:56,219 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41129 2023-02-08 08:53:56,230 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 08:53:56,230 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 08:53:57,277 WARN [Listener at localhost/42159] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:57,367 WARN [Listener at localhost/42159] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 08:53:57,391 WARN [Listener at localhost/42159] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:57,398 INFO [Listener at localhost/42159] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:57,402 INFO [Listener at localhost/42159] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/java.io.tmpdir/Jetty_localhost_42681_datanode____32om3c/webapp 2023-02-08 08:53:57,496 INFO [Listener at localhost/42159] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42681 2023-02-08 08:53:57,806 WARN [Listener at localhost/37597] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:57,819 WARN [Listener at localhost/37597] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 08:53:57,823 WARN [Listener at localhost/37597] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:57,825 INFO [Listener at localhost/37597] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:57,830 INFO [Listener at localhost/37597] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/java.io.tmpdir/Jetty_localhost_43505_datanode____h6egea/webapp 2023-02-08 08:53:57,927 INFO [Listener at localhost/37597] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43505 2023-02-08 08:53:57,936 WARN [Listener at localhost/34221] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:57,955 WARN [Listener at localhost/34221] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 08:53:57,960 WARN [Listener at localhost/34221] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:57,962 INFO [Listener at localhost/34221] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:57,966 INFO [Listener at localhost/34221] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/java.io.tmpdir/Jetty_localhost_38961_datanode____ljvbti/webapp 2023-02-08 08:53:58,111 INFO [Listener at localhost/34221] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38961 2023-02-08 08:53:58,119 WARN [Listener at localhost/33105] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:59,408 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6356118de250d6a8: Processing first storage report for DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f from datanode 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f 2023-02-08 08:53:59,410 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6356118de250d6a8: from storage DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f node DatanodeRegistration(127.0.0.1:39289, datanodeUuid=5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f, infoPort=36109, infoSecurePort=0, ipcPort=33105, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,410 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7bd976010275f624: Processing first storage report for DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354 from datanode 260f0410-1e75-4d9d-aa4e-3ba6e41b0476 2023-02-08 08:53:59,411 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7bd976010275f624: from storage DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354 node DatanodeRegistration(127.0.0.1:45451, datanodeUuid=260f0410-1e75-4d9d-aa4e-3ba6e41b0476, infoPort=46277, infoSecurePort=0, ipcPort=34221, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,411 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc1cac1746cffc: Processing first storage report for DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf from datanode 39265e67-bcfa-4780-a199-0487aaa87ffd 2023-02-08 08:53:59,411 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc1cac1746cffc: from storage DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf node DatanodeRegistration(127.0.0.1:44521, datanodeUuid=39265e67-bcfa-4780-a199-0487aaa87ffd, infoPort=35061, infoSecurePort=0, ipcPort=37597, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6356118de250d6a8: Processing first storage report for DS-d4afd06d-0f77-49c6-815e-e43ec1530b3b from datanode 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6356118de250d6a8: from storage DS-d4afd06d-0f77-49c6-815e-e43ec1530b3b node DatanodeRegistration(127.0.0.1:39289, datanodeUuid=5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f, infoPort=36109, infoSecurePort=0, ipcPort=33105, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7bd976010275f624: Processing first storage report for DS-97626cf3-4783-4300-8629-846a3c2bd440 from datanode 260f0410-1e75-4d9d-aa4e-3ba6e41b0476 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7bd976010275f624: from storage DS-97626cf3-4783-4300-8629-846a3c2bd440 node DatanodeRegistration(127.0.0.1:45451, datanodeUuid=260f0410-1e75-4d9d-aa4e-3ba6e41b0476, infoPort=46277, infoSecurePort=0, ipcPort=34221, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc1cac1746cffc: Processing first storage report for DS-ed16eaf6-8b1f-4971-90e2-bec9dac722bf from datanode 39265e67-bcfa-4780-a199-0487aaa87ffd 2023-02-08 08:53:59,412 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc1cac1746cffc: from storage DS-ed16eaf6-8b1f-4971-90e2-bec9dac722bf node DatanodeRegistration(127.0.0.1:44521, datanodeUuid=39265e67-bcfa-4780-a199-0487aaa87ffd, infoPort=35061, infoSecurePort=0, ipcPort=37597, storageInfo=lv=-57;cid=testClusterID;nsid=2076794466;c=1675846435525), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 08:53:59,423 DEBUG [Listener at localhost/33105] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1 2023-02-08 08:53:59,484 INFO [Listener at localhost/33105] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/zookeeper_0, clientPort=53039, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-08 08:53:59,498 INFO [Listener at localhost/33105] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53039 2023-02-08 08:53:59,508 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:59,511 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:00,169 INFO [Listener at localhost/33105] util.FSUtils(479): Created version file at hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb with version=8 2023-02-08 08:54:00,169 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/hbase-staging 2023-02-08 08:54:00,451 INFO [Listener at localhost/33105] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-08 08:54:00,902 INFO [Listener at localhost/33105] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 08:54:00,934 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:00,935 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:00,935 INFO [Listener at localhost/33105] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:54:00,935 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:00,935 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:54:01,075 INFO [Listener at localhost/33105] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:54:01,147 DEBUG [Listener at localhost/33105] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-08 08:54:01,244 INFO [Listener at localhost/33105] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33127 2023-02-08 08:54:01,255 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,257 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,278 INFO [Listener at localhost/33105] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33127 connecting to ZooKeeper ensemble=127.0.0.1:53039 2023-02-08 08:54:01,317 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:331270x0, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:54:01,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:33127-0x10033233bf30000 connected 2023-02-08 08:54:01,342 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:54:01,343 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:54:01,346 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:54:01,353 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33127 2023-02-08 08:54:01,354 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33127 2023-02-08 08:54:01,354 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33127 2023-02-08 08:54:01,355 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33127 2023-02-08 08:54:01,355 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33127 2023-02-08 08:54:01,360 INFO [Listener at localhost/33105] master.HMaster(439): hbase.rootdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb, hbase.cluster.distributed=false 2023-02-08 08:54:01,426 INFO [Listener at localhost/33105] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 08:54:01,427 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,427 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,427 INFO [Listener at localhost/33105] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:54:01,427 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,428 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:54:01,433 INFO [Listener at localhost/33105] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:54:01,436 INFO [Listener at localhost/33105] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44651 2023-02-08 08:54:01,438 INFO [Listener at localhost/33105] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 08:54:01,446 DEBUG [Listener at localhost/33105] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 08:54:01,447 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,449 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,450 INFO [Listener at localhost/33105] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44651 connecting to ZooKeeper ensemble=127.0.0.1:53039 2023-02-08 08:54:01,455 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:446510x0, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:54:01,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44651-0x10033233bf30001 connected 2023-02-08 08:54:01,456 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:54:01,457 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:54:01,458 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:54:01,459 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44651 2023-02-08 08:54:01,459 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44651 2023-02-08 08:54:01,459 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44651 2023-02-08 08:54:01,460 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44651 2023-02-08 08:54:01,460 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44651 2023-02-08 08:54:01,473 INFO [Listener at localhost/33105] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 08:54:01,473 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,474 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,474 INFO [Listener at localhost/33105] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:54:01,475 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,475 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:54:01,475 INFO [Listener at localhost/33105] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:54:01,477 INFO [Listener at localhost/33105] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36805 2023-02-08 08:54:01,477 INFO [Listener at localhost/33105] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 08:54:01,478 DEBUG [Listener at localhost/33105] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 08:54:01,479 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,482 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,484 INFO [Listener at localhost/33105] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36805 connecting to ZooKeeper ensemble=127.0.0.1:53039 2023-02-08 08:54:01,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:368050x0, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:54:01,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36805-0x10033233bf30002 connected 2023-02-08 08:54:01,488 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:54:01,489 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:54:01,490 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:54:01,492 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36805 2023-02-08 08:54:01,492 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36805 2023-02-08 08:54:01,492 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36805 2023-02-08 08:54:01,493 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36805 2023-02-08 08:54:01,493 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36805 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:54:01,505 INFO [Listener at localhost/33105] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:54:01,506 INFO [Listener at localhost/33105] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39055 2023-02-08 08:54:01,507 INFO [Listener at localhost/33105] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 08:54:01,508 DEBUG [Listener at localhost/33105] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 08:54:01,509 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,510 INFO [Listener at localhost/33105] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,511 INFO [Listener at localhost/33105] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39055 connecting to ZooKeeper ensemble=127.0.0.1:53039 2023-02-08 08:54:01,514 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:390550x0, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:54:01,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39055-0x10033233bf30003 connected 2023-02-08 08:54:01,515 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:54:01,516 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:54:01,517 DEBUG [Listener at localhost/33105] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:54:01,519 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39055 2023-02-08 08:54:01,519 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39055 2023-02-08 08:54:01,519 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39055 2023-02-08 08:54:01,520 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39055 2023-02-08 08:54:01,520 DEBUG [Listener at localhost/33105] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39055 2023-02-08 08:54:01,521 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:01,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 08:54:01,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:01,549 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:54:01,549 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:54:01,549 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:54:01,549 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:54:01,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:01,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 08:54:01,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33127,1675846440297 from backup master directory 2023-02-08 08:54:01,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 08:54:01,555 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:01,555 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 08:54:01,556 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:54:01,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:01,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-08 08:54:01,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-08 08:54:01,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/hbase.id with ID: 2856e76d-1897-4b68-bbfc-62eaba3e9755 2023-02-08 08:54:01,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:54:01,699 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:01,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x22458b66 to 127.0.0.1:53039 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:01,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c43dbe9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:01,800 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 08:54:01,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-08 08:54:01,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-08 08:54:01,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-08 08:54:01,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:54:01,825 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:54:01,826 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:54:01,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store-tmp 2023-02-08 08:54:01,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:01,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 08:54:01,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:54:01,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:54:01,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 08:54:01,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:54:01,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:54:01,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675846441913Disabling compacts and flushes for region at 1675846441914 (+1 ms)Disabling writes for close at 1675846441914Writing region close event to WAL at 1675846441914Closed at 1675846441914 2023-02-08 08:54:01,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/WALs/jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:01,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33127%2C1675846440297, suffix=, logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/WALs/jenkins-hbase4.apache.org,33127,1675846440297, archiveDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/oldWALs, maxLogs=10 2023-02-08 08:54:01,994 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK] 2023-02-08 08:54:01,994 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK] 2023-02-08 08:54:01,994 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK] 2023-02-08 08:54:02,005 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:54:02,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/WALs/jenkins-hbase4.apache.org,33127,1675846440297/jenkins-hbase4.apache.org%2C33127%2C1675846440297.1675846441945 2023-02-08 08:54:02,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK], DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK], DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK]] 2023-02-08 08:54:02,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:02,088 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:02,092 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,167 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,178 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-08 08:54:02,211 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-08 08:54:02,225 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:02,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:54:02,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:02,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63881155, jitterRate=-0.048096612095832825}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:02,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675846442124Initializing all the Stores at 1675846442126 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846442127 (+1 ms)Cleaning up temporary data from old regions at 1675846442241 (+114 ms)Cleaning up detritus from prior splits at 1675846442242 (+1 ms)Region opened successfully at 1675846442258 (+16 ms) 2023-02-08 08:54:02,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-08 08:54:02,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:02,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:02,286 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:02,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 08:54:02,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 42 msec 2023-02-08 08:54:02,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-08 08:54:02,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-08 08:54:02,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-08 08:54:02,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-08 08:54:02,406 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-08 08:54:02,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-08 08:54:02,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-08 08:54:02,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-08 08:54:02,423 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:02,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-08 08:54:02,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-08 08:54:02,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-08 08:54:02,446 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:54:02,446 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:54:02,446 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:54:02,446 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:54:02,447 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:02,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,33127,1675846440297, sessionid=0x10033233bf30000, setting cluster-up flag (Was=false) 2023-02-08 08:54:02,463 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:02,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-08 08:54:02,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:02,477 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:02,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-08 08:54:02,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:02,487 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.hbase-snapshot/.tmp 2023-02-08 08:54:02,526 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(952): ClusterId : 2856e76d-1897-4b68-bbfc-62eaba3e9755 2023-02-08 08:54:02,527 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(952): ClusterId : 2856e76d-1897-4b68-bbfc-62eaba3e9755 2023-02-08 08:54:02,526 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(952): ClusterId : 2856e76d-1897-4b68-bbfc-62eaba3e9755 2023-02-08 08:54:02,535 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 08:54:02,535 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 08:54:02,535 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 08:54:02,544 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 08:54:02,544 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 08:54:02,544 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 08:54:02,544 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 08:54:02,544 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 08:54:02,544 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 08:54:02,549 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 08:54:02,551 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 08:54:02,551 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ReadOnlyZKClient(139): Connect 0x2dd3b496 to 127.0.0.1:53039 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:02,552 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ReadOnlyZKClient(139): Connect 0x3cf43ab3 to 127.0.0.1:53039 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:02,553 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 08:54:02,562 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ReadOnlyZKClient(139): Connect 0x0ca17ad2 to 127.0.0.1:53039 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:02,584 DEBUG [RS:1;jenkins-hbase4:36805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23145f5c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:02,584 DEBUG [RS:2;jenkins-hbase4:39055] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2668536, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:02,584 DEBUG [RS:0;jenkins-hbase4:44651] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3233c35d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:02,584 DEBUG [RS:2;jenkins-hbase4:39055] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65911a2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:54:02,584 DEBUG [RS:1;jenkins-hbase4:36805] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@272bdfe6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:54:02,585 DEBUG [RS:0;jenkins-hbase4:44651] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@360c51c3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:54:02,617 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:39055 2023-02-08 08:54:02,617 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44651 2023-02-08 08:54:02,621 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:36805 2023-02-08 08:54:02,625 INFO [RS:1;jenkins-hbase4:36805] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 08:54:02,626 INFO [RS:1;jenkins-hbase4:36805] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 08:54:02,626 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 08:54:02,625 INFO [RS:2;jenkins-hbase4:39055] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 08:54:02,626 INFO [RS:2;jenkins-hbase4:39055] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 08:54:02,626 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 08:54:02,628 INFO [RS:0;jenkins-hbase4:44651] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 08:54:02,628 INFO [RS:0;jenkins-hbase4:44651] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 08:54:02,630 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 08:54:02,630 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33127,1675846440297 with isa=jenkins-hbase4.apache.org/172.31.14.131:36805, startcode=1675846441472 2023-02-08 08:54:02,631 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33127,1675846440297 with isa=jenkins-hbase4.apache.org/172.31.14.131:39055, startcode=1675846441504 2023-02-08 08:54:02,633 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33127,1675846440297 with isa=jenkins-hbase4.apache.org/172.31.14.131:44651, startcode=1675846441425 2023-02-08 08:54:02,656 DEBUG [RS:0;jenkins-hbase4:44651] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 08:54:02,656 DEBUG [RS:1;jenkins-hbase4:36805] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 08:54:02,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:02,656 DEBUG [RS:2;jenkins-hbase4:39055] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:54:02,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846472673 2023-02-08 08:54:02,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-08 08:54:02,689 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:02,689 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-08 08:54:02,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-08 08:54:02,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-08 08:54:02,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-08 08:54:02,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-08 08:54:02,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-08 08:54:02,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-08 08:54:02,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-08 08:54:02,719 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 08:54:02,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-08 08:54:02,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-08 08:54:02,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-08 08:54:02,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846442732,5,FailOnTimeoutGroup] 2023-02-08 08:54:02,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846442732,5,FailOnTimeoutGroup] 2023-02-08 08:54:02,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-08 08:54:02,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,741 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37143, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 08:54:02,741 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52467, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 08:54:02,741 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44515, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 08:54:02,757 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33127] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,785 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb 2023-02-08 08:54:02,785 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb 2023-02-08 08:54:02,785 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb 2023-02-08 08:54:02,785 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42159 2023-02-08 08:54:02,785 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42159 2023-02-08 08:54:02,785 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 08:54:02,785 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42159 2023-02-08 08:54:02,785 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 08:54:02,785 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 08:54:02,786 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:02,787 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:02,788 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb 2023-02-08 08:54:02,791 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:54:02,794 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ZKUtil(162): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,795 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ZKUtil(162): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,796 WARN [RS:1;jenkins-hbase4:36805] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:54:02,795 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ZKUtil(162): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,795 WARN [RS:0;jenkins-hbase4:44651] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:54:02,797 WARN [RS:2;jenkins-hbase4:39055] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:54:02,796 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39055,1675846441504] 2023-02-08 08:54:02,796 INFO [RS:1;jenkins-hbase4:36805] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:54:02,797 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36805,1675846441472] 2023-02-08 08:54:02,797 INFO [RS:2;jenkins-hbase4:39055] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:54:02,797 INFO [RS:0;jenkins-hbase4:44651] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:54:02,798 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,798 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,798 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,798 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44651,1675846441425] 2023-02-08 08:54:02,815 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ZKUtil(162): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,815 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ZKUtil(162): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,816 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ZKUtil(162): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,816 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ZKUtil(162): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,816 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ZKUtil(162): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,816 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ZKUtil(162): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,816 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ZKUtil(162): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,817 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ZKUtil(162): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,817 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ZKUtil(162): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,822 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:02,825 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 08:54:02,828 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/info 2023-02-08 08:54:02,828 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 08:54:02,829 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 08:54:02,830 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:02,829 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 08:54:02,829 DEBUG [RS:0;jenkins-hbase4:44651] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 08:54:02,830 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 08:54:02,833 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:54:02,834 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 08:54:02,835 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:02,835 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 08:54:02,837 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/table 2023-02-08 08:54:02,838 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 08:54:02,839 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:02,841 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740 2023-02-08 08:54:02,842 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740 2023-02-08 08:54:02,842 INFO [RS:1;jenkins-hbase4:36805] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 08:54:02,843 INFO [RS:2;jenkins-hbase4:39055] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 08:54:02,842 INFO [RS:0;jenkins-hbase4:44651] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 08:54:02,847 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 08:54:02,850 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 08:54:02,863 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:02,864 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75173175, jitterRate=0.12016759812831879}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 08:54:02,864 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675846442823Initializing all the Stores at 1675846442824 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846442824Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846442825 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846442825Cleaning up temporary data from old regions at 1675846442845 (+20 ms)Cleaning up detritus from prior splits at 1675846442845Region opened successfully at 1675846442864 (+19 ms) 2023-02-08 08:54:02,864 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 08:54:02,864 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 08:54:02,864 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 08:54:02,864 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 08:54:02,864 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 08:54:02,865 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 08:54:02,866 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675846442864Disabling compacts and flushes for region at 1675846442864Disabling writes for close at 1675846442864Writing region close event to WAL at 1675846442865 (+1 ms)Closed at 1675846442865 2023-02-08 08:54:02,872 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:02,872 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-08 08:54:02,873 INFO [RS:1;jenkins-hbase4:36805] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 08:54:02,878 INFO [RS:0;jenkins-hbase4:44651] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 08:54:02,873 INFO [RS:2;jenkins-hbase4:39055] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 08:54:02,884 INFO [RS:0;jenkins-hbase4:44651] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:54:02,884 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,884 INFO [RS:2;jenkins-hbase4:39055] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:54:02,885 INFO [RS:1;jenkins-hbase4:36805] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:54:02,885 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,886 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 08:54:02,886 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,887 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 08:54:02,889 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 08:54:02,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-08 08:54:02,896 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,896 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,896 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,896 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,896 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:54:02,898 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:54:02,898 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,897 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:0;jenkins-hbase4:44651] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:2;jenkins-hbase4:39055] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,898 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:54:02,899 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,899 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,899 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,899 DEBUG [RS:1;jenkins-hbase4:36805] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:54:02,901 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,901 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,902 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,902 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,902 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,902 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,904 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,904 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,904 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-08 08:54:02,910 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 08:54:02,923 INFO [RS:0;jenkins-hbase4:44651] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 08:54:02,923 INFO [RS:1;jenkins-hbase4:36805] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 08:54:02,923 INFO [RS:2;jenkins-hbase4:39055] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 08:54:02,926 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39055,1675846441504-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,926 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36805,1675846441472-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,926 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44651,1675846441425-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:02,949 INFO [RS:1;jenkins-hbase4:36805] regionserver.Replication(203): jenkins-hbase4.apache.org,36805,1675846441472 started 2023-02-08 08:54:02,949 INFO [RS:2;jenkins-hbase4:39055] regionserver.Replication(203): jenkins-hbase4.apache.org,39055,1675846441504 started 2023-02-08 08:54:02,949 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36805,1675846441472, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36805, sessionid=0x10033233bf30002 2023-02-08 08:54:02,950 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39055,1675846441504, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39055, sessionid=0x10033233bf30003 2023-02-08 08:54:02,950 INFO [RS:0;jenkins-hbase4:44651] regionserver.Replication(203): jenkins-hbase4.apache.org,44651,1675846441425 started 2023-02-08 08:54:02,950 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,44651,1675846441425, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44651, sessionid=0x10033233bf30001 2023-02-08 08:54:02,950 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 08:54:02,950 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 08:54:02,950 DEBUG [RS:2;jenkins-hbase4:39055] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,950 DEBUG [RS:0;jenkins-hbase4:44651] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,950 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 08:54:02,951 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44651,1675846441425' 2023-02-08 08:54:02,951 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39055,1675846441504' 2023-02-08 08:54:02,951 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:02,951 DEBUG [RS:1;jenkins-hbase4:36805] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,951 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:02,951 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36805,1675846441472' 2023-02-08 08:54:02,951 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:02,952 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:02,952 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:02,952 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:02,952 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 08:54:02,953 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 08:54:02,953 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 08:54:02,953 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 08:54:02,953 DEBUG [RS:0;jenkins-hbase4:44651] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:02,953 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 08:54:02,953 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 08:54:02,953 DEBUG [RS:1;jenkins-hbase4:36805] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:02,953 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36805,1675846441472' 2023-02-08 08:54:02,954 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:54:02,953 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44651,1675846441425' 2023-02-08 08:54:02,953 DEBUG [RS:2;jenkins-hbase4:39055] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:02,954 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39055,1675846441504' 2023-02-08 08:54:02,954 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:54:02,954 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:54:02,954 DEBUG [RS:1;jenkins-hbase4:36805] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:54:02,954 DEBUG [RS:0;jenkins-hbase4:44651] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:54:02,954 DEBUG [RS:2;jenkins-hbase4:39055] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:54:02,955 DEBUG [RS:1;jenkins-hbase4:36805] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 08:54:02,955 INFO [RS:1;jenkins-hbase4:36805] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 08:54:02,955 DEBUG [RS:0;jenkins-hbase4:44651] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 08:54:02,955 INFO [RS:1;jenkins-hbase4:36805] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 08:54:02,955 INFO [RS:0;jenkins-hbase4:44651] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 08:54:02,956 INFO [RS:0;jenkins-hbase4:44651] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 08:54:02,956 DEBUG [RS:2;jenkins-hbase4:39055] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 08:54:02,956 INFO [RS:2;jenkins-hbase4:39055] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 08:54:02,956 INFO [RS:2;jenkins-hbase4:39055] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 08:54:03,062 DEBUG [jenkins-hbase4:33127] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-08 08:54:03,066 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:03,067 INFO [RS:2;jenkins-hbase4:39055] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39055%2C1675846441504, suffix=, logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,39055,1675846441504, archiveDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs, maxLogs=32 2023-02-08 08:54:03,067 INFO [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36805%2C1675846441472, suffix=, logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472, archiveDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs, maxLogs=32 2023-02-08 08:54:03,067 INFO [RS:0;jenkins-hbase4:44651] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44651%2C1675846441425, suffix=, logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,44651,1675846441425, archiveDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs, maxLogs=32 2023-02-08 08:54:03,075 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:03,075 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:03,075 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:03,075 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:03,079 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36805,1675846441472, state=OPENING 2023-02-08 08:54:03,089 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-08 08:54:03,091 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:03,096 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 08:54:03,097 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK] 2023-02-08 08:54:03,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36805,1675846441472}] 2023-02-08 08:54:03,103 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK] 2023-02-08 08:54:03,103 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK] 2023-02-08 08:54:03,103 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK] 2023-02-08 08:54:03,104 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK] 2023-02-08 08:54:03,106 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK] 2023-02-08 08:54:03,107 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK] 2023-02-08 08:54:03,107 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK] 2023-02-08 08:54:03,108 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK] 2023-02-08 08:54:03,118 INFO [RS:2;jenkins-hbase4:39055] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,39055,1675846441504/jenkins-hbase4.apache.org%2C39055%2C1675846441504.1675846443073 2023-02-08 08:54:03,119 DEBUG [RS:2;jenkins-hbase4:39055] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK], DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK], DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK]] 2023-02-08 08:54:03,119 INFO [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472/jenkins-hbase4.apache.org%2C36805%2C1675846441472.1675846443073 2023-02-08 08:54:03,119 DEBUG [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK], DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK], DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK]] 2023-02-08 08:54:03,120 INFO [RS:0;jenkins-hbase4:44651] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,44651,1675846441425/jenkins-hbase4.apache.org%2C44651%2C1675846441425.1675846443073 2023-02-08 08:54:03,120 DEBUG [RS:0;jenkins-hbase4:44651] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK], DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK], DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK]] 2023-02-08 08:54:03,304 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:03,307 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:54:03,310 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53438, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:54:03,323 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-08 08:54:03,323 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:54:03,328 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36805%2C1675846441472.meta, suffix=.meta, logDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472, archiveDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs, maxLogs=32 2023-02-08 08:54:03,357 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK] 2023-02-08 08:54:03,359 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK] 2023-02-08 08:54:03,359 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK] 2023-02-08 08:54:03,365 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472/jenkins-hbase4.apache.org%2C36805%2C1675846441472.meta.1675846443330.meta 2023-02-08 08:54:03,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45451,DS-bd1264c7-ee1f-4ffc-8d42-b133a9dd1354,DISK], DatanodeInfoWithStorage[127.0.0.1:44521,DS-910602e1-d2c4-4179-bbe7-c9e15083e2cf,DISK], DatanodeInfoWithStorage[127.0.0.1:39289,DS-57efb5cb-3ae8-447f-a9a4-7dc483eb9b2f,DISK]] 2023-02-08 08:54:03,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:03,369 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:03,370 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-08 08:54:03,387 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-08 08:54:03,390 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-08 08:54:03,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-08 08:54:03,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:03,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-08 08:54:03,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-08 08:54:03,397 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 08:54:03,399 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/info 2023-02-08 08:54:03,400 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/info 2023-02-08 08:54:03,400 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 08:54:03,402 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:03,402 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 08:54:03,404 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:54:03,404 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:54:03,404 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 08:54:03,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:03,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 08:54:03,407 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/table 2023-02-08 08:54:03,407 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/table 2023-02-08 08:54:03,407 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 08:54:03,408 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:03,410 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740 2023-02-08 08:54:03,412 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740 2023-02-08 08:54:03,415 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 08:54:03,417 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 08:54:03,419 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67716825, jitterRate=0.009059324860572815}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 08:54:03,419 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675846443395Writing region info on filesystem at 1675846443395Initializing all the Stores at 1675846443397 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846443397Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846443397Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846443397Cleaning up temporary data from old regions at 1675846443414 (+17 ms)Cleaning up detritus from prior splits at 1675846443414Running coprocessor post-open hooks at 1675846443419 (+5 ms)Region opened successfully at 1675846443419 2023-02-08 08:54:03,434 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675846443295 2023-02-08 08:54:03,453 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-08 08:54:03,455 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-08 08:54:03,455 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36805,1675846441472, state=OPEN 2023-02-08 08:54:03,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-08 08:54:03,459 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 08:54:03,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-08 08:54:03,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36805,1675846441472 in 359 msec 2023-02-08 08:54:03,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-08 08:54:03,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 575 msec 2023-02-08 08:54:03,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 930 msec 2023-02-08 08:54:03,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675846441545, completionTime=-1 2023-02-08 08:54:03,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-08 08:54:03,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:03,536 DEBUG [hconnection-0x5d1b4a5f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:03,539 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37826, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:03,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:03,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846503561 2023-02-08 08:54:03,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846563561 2023-02-08 08:54:03,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 87 msec 2023-02-08 08:54:03,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33127,1675846440297-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:03,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33127,1675846440297-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:03,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33127,1675846440297-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:03,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33127, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:03,588 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-08 08:54:03,594 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-08 08:54:03,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-08 08:54:03,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 08:54:03,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:03,615 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:03,618 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:03,641 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:03,643 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf empty. 2023-02-08 08:54:03,644 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:03,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-08 08:54:03,682 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:03,684 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9a7e5b1dad112219d73dc56e1f1e8ebf, NAME => 'hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 9a7e5b1dad112219d73dc56e1f1e8ebf, disabling compactions & flushes 2023-02-08 08:54:03,705 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. after waiting 0 ms 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:03,705 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:03,705 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 9a7e5b1dad112219d73dc56e1f1e8ebf: Waiting for close lock at 1675846443705Disabling compacts and flushes for region at 1675846443705Disabling writes for close at 1675846443705Writing region close event to WAL at 1675846443705Closed at 1675846443705 2023-02-08 08:54:03,710 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:03,725 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846443713"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846443713"}]},"ts":"1675846443713"} 2023-02-08 08:54:03,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:03,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:03,761 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846443756"}]},"ts":"1675846443756"} 2023-02-08 08:54:03,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-08 08:54:03,771 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:03,772 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:03,772 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:03,772 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:03,772 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:03,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9a7e5b1dad112219d73dc56e1f1e8ebf, ASSIGN}] 2023-02-08 08:54:03,778 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=9a7e5b1dad112219d73dc56e1f1e8ebf, ASSIGN 2023-02-08 08:54:03,779 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=9a7e5b1dad112219d73dc56e1f1e8ebf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:03,931 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:03,932 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:03,933 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846443932"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846443932"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846443932"}]},"ts":"1675846443932"} 2023-02-08 08:54:03,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 9a7e5b1dad112219d73dc56e1f1e8ebf, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:04,091 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:04,092 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:54:04,094 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40160, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:54:04,100 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:04,100 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a7e5b1dad112219d73dc56e1f1e8ebf, NAME => 'hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:04,101 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:04,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:04,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,102 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,104 INFO [StoreOpener-9a7e5b1dad112219d73dc56e1f1e8ebf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,106 DEBUG [StoreOpener-9a7e5b1dad112219d73dc56e1f1e8ebf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/info 2023-02-08 08:54:04,106 DEBUG [StoreOpener-9a7e5b1dad112219d73dc56e1f1e8ebf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/info 2023-02-08 08:54:04,107 INFO [StoreOpener-9a7e5b1dad112219d73dc56e1f1e8ebf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a7e5b1dad112219d73dc56e1f1e8ebf columnFamilyName info 2023-02-08 08:54:04,107 INFO [StoreOpener-9a7e5b1dad112219d73dc56e1f1e8ebf-1] regionserver.HStore(310): Store=9a7e5b1dad112219d73dc56e1f1e8ebf/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:04,109 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,110 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,114 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:54:04,118 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:04,119 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a7e5b1dad112219d73dc56e1f1e8ebf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61004254, jitterRate=-0.09096577763557434}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:04,119 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a7e5b1dad112219d73dc56e1f1e8ebf: Running coprocessor pre-open hook at 1675846444102Writing region info on filesystem at 1675846444102Initializing all the Stores at 1675846444104 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846444104Cleaning up temporary data from old regions at 1675846444110 (+6 ms)Cleaning up detritus from prior splits at 1675846444111 (+1 ms)Running coprocessor post-open hooks at 1675846444119 (+8 ms)Region opened successfully at 1675846444119 2023-02-08 08:54:04,122 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf., pid=6, masterSystemTime=1675846444091 2023-02-08 08:54:04,126 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:04,126 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:54:04,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:04,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846444127"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846444127"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846444127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846444127"}]},"ts":"1675846444127"} 2023-02-08 08:54:04,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-08 08:54:04,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 9a7e5b1dad112219d73dc56e1f1e8ebf, server=jenkins-hbase4.apache.org,39055,1675846441504 in 194 msec 2023-02-08 08:54:04,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-08 08:54:04,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=9a7e5b1dad112219d73dc56e1f1e8ebf, ASSIGN in 361 msec 2023-02-08 08:54:04,140 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:04,140 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846444140"}]},"ts":"1675846444140"} 2023-02-08 08:54:04,143 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-08 08:54:04,146 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:04,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 542 msec 2023-02-08 08:54:04,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-08 08:54:04,218 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:54:04,218 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:04,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:04,241 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40176, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:04,256 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:04,282 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:54:04,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 40 msec 2023-02-08 08:54:04,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:04,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:54:04,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-08 08:54:04,326 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-08 08:54:04,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-08 08:54:04,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.772sec 2023-02-08 08:54:04,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-08 08:54:04,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-08 08:54:04,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-08 08:54:04,333 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33127,1675846440297-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-08 08:54:04,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33127,1675846440297-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-08 08:54:04,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-08 08:54:04,432 DEBUG [Listener at localhost/33105] zookeeper.ReadOnlyZKClient(139): Connect 0x58b3884f to 127.0.0.1:53039 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:04,440 DEBUG [Listener at localhost/33105] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16e5d675, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:04,456 DEBUG [hconnection-0x4c175139-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:04,468 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37834, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:04,478 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:54:04,507 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=361, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=168, AvailableMemoryMB=4629 2023-02-08 08:54:04,512 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:04,513 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:04,513 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:04,521 DEBUG [Listener at localhost/33105] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:04,524 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51158, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:04,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:04,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:54:04,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:04,542 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:04,544 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:04,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:04,549 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,550 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b empty. 2023-02-08 08:54:04,552 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,552 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 08:54:04,573 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:04,575 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 366926f8e963611bc9739ee35ac1108b, NAME => 'testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 366926f8e963611bc9739ee35ac1108b, disabling compactions & flushes 2023-02-08 08:54:04,626 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. after waiting 0 ms 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,626 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,626 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 366926f8e963611bc9739ee35ac1108b: Waiting for close lock at 1675846444626Disabling compacts and flushes for region at 1675846444626Disabling writes for close at 1675846444626Writing region close event to WAL at 1675846444626Closed at 1675846444626 2023-02-08 08:54:04,631 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:04,632 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846444632"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846444632"}]},"ts":"1675846444632"} 2023-02-08 08:54:04,635 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:04,636 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:04,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846444636"}]},"ts":"1675846444636"} 2023-02-08 08:54:04,639 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-08 08:54:04,643 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:04,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:04,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:04,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:04,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:04,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, ASSIGN}] 2023-02-08 08:54:04,648 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, ASSIGN 2023-02-08 08:54:04,649 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:04,800 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:04,801 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=366926f8e963611bc9739ee35ac1108b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:04,801 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846444801"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846444801"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846444801"}]},"ts":"1675846444801"} 2023-02-08 08:54:04,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 366926f8e963611bc9739ee35ac1108b, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:04,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 366926f8e963611bc9739ee35ac1108b, NAME => 'testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:04,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:04,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:04,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,966 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,968 DEBUG [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1 2023-02-08 08:54:04,968 DEBUG [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1 2023-02-08 08:54:04,969 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 366926f8e963611bc9739ee35ac1108b columnFamilyName cf1 2023-02-08 08:54:04,970 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] regionserver.HStore(310): Store=366926f8e963611bc9739ee35ac1108b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:04,970 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,972 DEBUG [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2 2023-02-08 08:54:04,972 DEBUG [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2 2023-02-08 08:54:04,972 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 366926f8e963611bc9739ee35ac1108b columnFamilyName cf2 2023-02-08 08:54:04,973 INFO [StoreOpener-366926f8e963611bc9739ee35ac1108b-1] regionserver.HStore(310): Store=366926f8e963611bc9739ee35ac1108b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:04,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:04,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:04,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:04,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 366926f8e963611bc9739ee35ac1108b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63274741, jitterRate=-0.05713288486003876}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:04,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 366926f8e963611bc9739ee35ac1108b: Running coprocessor pre-open hook at 1675846444964Writing region info on filesystem at 1675846444964Initializing all the Stores at 1675846444966 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846444966Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846444966Cleaning up temporary data from old regions at 1675846444977 (+11 ms)Cleaning up detritus from prior splits at 1675846444977Running coprocessor post-open hooks at 1675846444985 (+8 ms)Region opened successfully at 1675846444985 2023-02-08 08:54:04,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b., pid=11, masterSystemTime=1675846444957 2023-02-08 08:54:04,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:04,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=366926f8e963611bc9739ee35ac1108b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:04,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846444989"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846444989"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846444989"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846444989"}]},"ts":"1675846444989"} 2023-02-08 08:54:04,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-08 08:54:04,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 366926f8e963611bc9739ee35ac1108b, server=jenkins-hbase4.apache.org,39055,1675846441504 in 189 msec 2023-02-08 08:54:04,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-08 08:54:05,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, ASSIGN in 352 msec 2023-02-08 08:54:05,001 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:05,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846445001"}]},"ts":"1675846445001"} 2023-02-08 08:54:05,003 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-08 08:54:05,006 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:05,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 465 msec 2023-02-08 08:54:05,048 DEBUG [Listener at localhost/33105] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:05,052 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40188, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:05,071 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,071 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:05,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,082 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,082 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,083 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:05,083 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:05,083 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,083 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,085 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,085 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,085 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,085 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,089 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,089 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,089 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,090 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,091 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,091 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,091 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,091 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:05,091 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:05,091 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,092 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:05,093 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:54:05,094 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,095 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,164 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/acba32b91bfc465ea90a39bbde72f925 2023-02-08 08:54:05,221 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/defd7019e39c4ac2b799f252daa3440e 2023-02-08 08:54:05,235 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/acba32b91bfc465ea90a39bbde72f925 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925 2023-02-08 08:54:05,245 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:05,248 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/defd7019e39c4ac2b799f252daa3440e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e 2023-02-08 08:54:05,258 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:05,260 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 167ms, sequenceid=6, compaction requested=false 2023-02-08 08:54:05,262 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,264 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:05,264 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,264 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,269 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,269 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:05,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38f02441[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:05,274 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:05,274 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,278 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,279 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,282 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,284 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:05,290 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:05,290 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:05,290 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:05,290 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,290 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,294 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,294 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,295 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,297 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,298 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,298 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,309 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,310 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:05,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,311 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,314 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,314 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,314 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:05,314 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:05,414 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:05,415 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,415 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:05,443 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,443 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:05,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:05,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,447 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,447 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,448 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:05,448 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:05,448 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,448 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,451 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,451 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,451 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,454 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,455 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:05,456 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:05,456 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,456 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:05,456 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:05,458 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,458 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,458 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,458 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,459 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,459 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,481 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/e26107add432444bb0ad121abe2dc374 2023-02-08 08:54:05,513 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/f22d695fbd5146a1ad0ee800b22e86db 2023-02-08 08:54:05,523 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/e26107add432444bb0ad121abe2dc374 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374 2023-02-08 08:54:05,531 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:05,533 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/f22d695fbd5146a1ad0ee800b22e86db as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db 2023-02-08 08:54:05,540 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:05,542 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 86ms, sequenceid=19, compaction requested=false 2023-02-08 08:54:05,542 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:05,542 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,545 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,545 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,545 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,545 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:05,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b94cf3f[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:05,548 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:05,548 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,551 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,551 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:05,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,553 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:05,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:05,559 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:05,559 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:05,559 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:05,560 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,560 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,562 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,563 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,564 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,565 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,567 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,568 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,568 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,578 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:05,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,578 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,580 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:05,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,580 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:05,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,681 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:05,681 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,681 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:05,705 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,705 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:05,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:05,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,708 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,708 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,709 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:05,709 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:05,710 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,710 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,711 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,711 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,711 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,711 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,712 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,712 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,714 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,714 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,716 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,717 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,718 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:05,718 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:05,718 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,718 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:05,718 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:05,719 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,720 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,720 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,721 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,737 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/22c481f5e9994ce28dfe389e7411458e 2023-02-08 08:54:05,761 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/a9a297fc793d4d45826bb1c7a53660dd 2023-02-08 08:54:05,770 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/22c481f5e9994ce28dfe389e7411458e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e 2023-02-08 08:54:05,777 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:05,778 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/a9a297fc793d4d45826bb1c7a53660dd as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd 2023-02-08 08:54:05,785 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:05,786 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 68ms, sequenceid=32, compaction requested=true 2023-02-08 08:54:05,786 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:05,787 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,787 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:05,787 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:05,787 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,787 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,787 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,790 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,790 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,791 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:05,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bc46726[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:05,794 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:05,794 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,795 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,795 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:05,796 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:05,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,800 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,800 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:05,800 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,800 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,800 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,800 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,801 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:05,807 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:05,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:05,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:05,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,807 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,810 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,811 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,812 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,815 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,815 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,816 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,819 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,819 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,820 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,820 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:05,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,821 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,921 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:05,921 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,921 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:05,945 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,945 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:05,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:05,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,948 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:05,948 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,949 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:05,949 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:05,950 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,950 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,951 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,951 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:05,951 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:05,954 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,954 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,954 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:05,955 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:05,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:05,955 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:05,956 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:05,957 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,958 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,958 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,958 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,959 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:05,959 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,974 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/e6b0a964acba4a5cb897762803aeba28 2023-02-08 08:54:05,997 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/e37b1f7e31f24a629f47f03e39ed69db 2023-02-08 08:54:06,004 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/e6b0a964acba4a5cb897762803aeba28 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28 2023-02-08 08:54:06,011 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:06,013 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/e37b1f7e31f24a629f47f03e39ed69db as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db 2023-02-08 08:54:06,019 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:06,020 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 64ms, sequenceid=45, compaction requested=true 2023-02-08 08:54:06,021 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:06,021 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,021 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:06,021 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:06,021 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,021 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,021 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,026 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,026 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,026 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,026 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:06,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@292c8f80[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:06,029 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:06,029 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,030 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,031 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,031 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:06,030 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:06,030 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:06,033 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,033 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,033 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,033 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,034 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,034 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,034 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,035 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,035 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,035 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:06,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:06,039 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:06,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:06,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:06,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,039 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,041 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,042 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,042 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,045 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,045 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,045 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,050 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:06,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,050 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,051 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,051 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:06,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,051 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,051 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:06,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,051 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,052 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,152 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:06,152 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,152 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:06,173 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,173 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:06,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:06,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,177 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:06,177 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,177 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:06,177 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:06,178 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,178 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,179 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,179 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,179 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,179 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,180 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,180 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:06,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,185 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,185 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,185 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,185 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,186 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,186 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,186 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,186 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,186 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,187 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,187 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,187 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,189 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,189 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:06,189 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:06,189 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,189 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,189 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,189 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:06,189 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:06,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,192 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,192 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,206 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/d91ca415e785443fa9c6c7b018793d7b 2023-02-08 08:54:06,236 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/b89b2c5ef81742be9737e109daea622a 2023-02-08 08:54:06,244 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/d91ca415e785443fa9c6c7b018793d7b as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b 2023-02-08 08:54:06,251 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:06,252 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/b89b2c5ef81742be9737e109daea622a as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a 2023-02-08 08:54:06,259 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:06,261 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 71ms, sequenceid=58, compaction requested=true 2023-02-08 08:54:06,261 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:06,261 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,261 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,262 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,263 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,264 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,264 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,264 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10c774c8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:06,266 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:06,266 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,268 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,268 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:06,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:06,269 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,270 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,270 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,271 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,271 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,271 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,271 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,272 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,272 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,272 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:06,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:06,276 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:06,276 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:06,276 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:06,276 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,276 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:06,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,278 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,279 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,279 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,281 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,281 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,282 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,288 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,289 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:06,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:06,288 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,289 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,289 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,289 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:06,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,290 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,390 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:06,390 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,391 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:06,411 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,411 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:06,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:06,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,415 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:06,415 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,416 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 08:54:06,416 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 08:54:06,416 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,416 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,417 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,417 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,417 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,417 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,418 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,418 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,419 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,419 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,422 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 08:54:06,422 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:06,422 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,422 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:06,422 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,422 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. started... 2023-02-08 08:54:06,422 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,422 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:06,423 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,425 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,425 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,425 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,426 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,426 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,438 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/6a5771cd22c74e9e93e4ff5568b229ca 2023-02-08 08:54:06,461 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/3fecc7b0d29a401eb5aab7626eae0285 2023-02-08 08:54:06,468 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/6a5771cd22c74e9e93e4ff5568b229ca as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca 2023-02-08 08:54:06,475 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:06,476 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/3fecc7b0d29a401eb5aab7626eae0285 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285 2023-02-08 08:54:06,482 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:06,484 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 62ms, sequenceid=71, compaction requested=true 2023-02-08 08:54:06,484 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 366926f8e963611bc9739ee35ac1108b: 2023-02-08 08:54:06,484 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,484 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:06,485 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:06,485 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,485 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 08:54:06,485 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 08:54:06,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,487 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,487 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,487 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 08:54:06,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d07acd8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:06,489 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:06,489 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,490 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,490 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,490 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,491 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 08:54:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,493 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,495 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 08:54:06,495 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,495 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 08:54:06,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:54:06,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:06,498 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 08:54:06,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:06,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:06,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,498 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,500 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,500 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,501 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,504 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,505 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,506 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,506 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 08:54:06,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:06,509 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,509 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,510 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:06,510 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:06,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:06,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,511 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 08:54:06,611 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:06,611 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,611 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:06,634 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,640 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:06,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:06,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:06,641 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:06,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, UNASSIGN}] 2023-02-08 08:54:06,649 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, UNASSIGN 2023-02-08 08:54:06,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=366926f8e963611bc9739ee35ac1108b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:06,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846446650"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846446650"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846446650"}]},"ts":"1675846446650"} 2023-02-08 08:54:06,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 366926f8e963611bc9739ee35ac1108b, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:06,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 366926f8e963611bc9739ee35ac1108b, disabling compactions & flushes 2023-02-08 08:54:06,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. after waiting 0 ms 2023-02-08 08:54:06,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 366926f8e963611bc9739ee35ac1108b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:54:06,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/58afdbaf25874a1d93931202a3da6cb4 2023-02-08 08:54:06,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 2023-02-08 08:54:06,859 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf1/58afdbaf25874a1d93931202a3da6cb4 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4 2023-02-08 08:54:06,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:06,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.tmp/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 2023-02-08 08:54:06,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:06,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 64ms, sequenceid=82, compaction requested=true 2023-02-08 08:54:06,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:54:06,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:06,898 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. 2023-02-08 08:54:06,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 366926f8e963611bc9739ee35ac1108b: Waiting for close lock at 1675846446813Running coprocessor pre-close hooks at 1675846446813Disabling compacts and flushes for region at 1675846446813Disabling writes for close at 1675846446813Obtaining lock to block concurrent updates at 1675846446813Preparing flush snapshotting stores in 366926f8e963611bc9739ee35ac1108b at 1675846446813Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846446813Flushing stores of testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. at 1675846446814 (+1 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf1: creating writer at 1675846446815 (+1 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf1: appending metadata at 1675846446818 (+3 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf1: closing flushed file at 1675846446818Flushing 366926f8e963611bc9739ee35ac1108b/cf2: creating writer at 1675846446838 (+20 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf2: appending metadata at 1675846446840 (+2 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf2: closing flushed file at 1675846446840Flushing 366926f8e963611bc9739ee35ac1108b/cf1: reopening flushed file at 1675846446860 (+20 ms)Flushing 366926f8e963611bc9739ee35ac1108b/cf2: reopening flushed file at 1675846446870 (+10 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 366926f8e963611bc9739ee35ac1108b in 64ms, sequenceid=82, compaction requested=true at 1675846446877 (+7 ms)Writing region close event to WAL at 1675846446892 (+15 ms)Running coprocessor post-close hooks at 1675846446897 (+5 ms)Closed at 1675846446898 (+1 ms) 2023-02-08 08:54:06,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,901 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=366926f8e963611bc9739ee35ac1108b, regionState=CLOSED 2023-02-08 08:54:06,902 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846446901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846446901"}]},"ts":"1675846446901"} 2023-02-08 08:54:06,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-08 08:54:06,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 366926f8e963611bc9739ee35ac1108b, server=jenkins-hbase4.apache.org,39055,1675846441504 in 251 msec 2023-02-08 08:54:06,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-08 08:54:06,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=366926f8e963611bc9739ee35ac1108b, UNASSIGN in 258 msec 2023-02-08 08:54:06,927 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=366926f8e963611bc9739ee35ac1108b, threads=14 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,928 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,929 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,930 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,930 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:06,958 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11, top=true 2023-02-08 08:54:06,959 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a, top=true 2023-02-08 08:54:06,960 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca, top=true 2023-02-08 08:54:06,963 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285, top=true 2023-02-08 08:54:06,963 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b, top=true 2023-02-08 08:54:06,969 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4, top=true 2023-02-08 08:54:07,016 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,016 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,017 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285 for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,017 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4 for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/.splits/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11 for child: 0c013ea68aeb4bc9d7e5174c5e923659, parent: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,032 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,115 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,133 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,136 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,156 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,157 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,157 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,159 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,160 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925 for region: 366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:07,160 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 366926f8e963611bc9739ee35ac1108b Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:07,233 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:07,236 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:07,239 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675846447239"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675846447239"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675846447239"}]},"ts":"1675846447239"} 2023-02-08 08:54:07,239 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447239"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846447239"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846447239"}]},"ts":"1675846447239"} 2023-02-08 08:54:07,240 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447239"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846447239"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846447239"}]},"ts":"1675846447239"} 2023-02-08 08:54:07,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, ASSIGN}] 2023-02-08 08:54:07,289 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, ASSIGN 2023-02-08 08:54:07,289 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:07,290 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, ASSIGN 2023-02-08 08:54:07,291 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:07,440 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:54:07,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0c013ea68aeb4bc9d7e5174c5e923659, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:07,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447441"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846447441"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846447441"}]},"ts":"1675846447441"} 2023-02-08 08:54:07,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 0c013ea68aeb4bc9d7e5174c5e923659, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:07,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=67cb0b0aee3415fbb18966a88c15b8b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:07,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447448"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846447448"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846447448"}]},"ts":"1675846447448"} 2023-02-08 08:54:07,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 67cb0b0aee3415fbb18966a88c15b8b5, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:07,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0c013ea68aeb4bc9d7e5174c5e923659, NAME => 'testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.', STARTKEY => '26', ENDKEY => ''} 2023-02-08 08:54:07,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:07,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:07,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,603 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,605 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1 2023-02-08 08:54:07,605 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1 2023-02-08 08:54:07,605 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c013ea68aeb4bc9d7e5174c5e923659 columnFamilyName cf1 2023-02-08 08:54:07,637 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-top 2023-02-08 08:54:07,644 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-top 2023-02-08 08:54:07,651 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-top 2023-02-08 08:54:07,657 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-top 2023-02-08 08:54:07,663 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4 2023-02-08 08:54:07,668 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca 2023-02-08 08:54:07,672 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b 2023-02-08 08:54:07,673 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(310): Store=0c013ea68aeb4bc9d7e5174c5e923659/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:07,673 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,674 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2 2023-02-08 08:54:07,674 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2 2023-02-08 08:54:07,675 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c013ea68aeb4bc9d7e5174c5e923659 columnFamilyName cf2 2023-02-08 08:54:07,690 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-top 2023-02-08 08:54:07,696 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-top 2023-02-08 08:54:07,702 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-top 2023-02-08 08:54:07,707 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-top 2023-02-08 08:54:07,712 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285 2023-02-08 08:54:07,717 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11 2023-02-08 08:54:07,722 DEBUG [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a 2023-02-08 08:54:07,722 INFO [StoreOpener-0c013ea68aeb4bc9d7e5174c5e923659-1] regionserver.HStore(310): Store=0c013ea68aeb4bc9d7e5174c5e923659/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:07,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:07,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:07,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0c013ea68aeb4bc9d7e5174c5e923659; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73616798, jitterRate=0.09697577357292175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:07,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0c013ea68aeb4bc9d7e5174c5e923659: Running coprocessor pre-open hook at 1675846447602Writing region info on filesystem at 1675846447602Initializing all the Stores at 1675846447603 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846447603Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846447603Cleaning up temporary data from old regions at 1675846447727 (+124 ms)Cleaning up detritus from prior splits at 1675846447727Running coprocessor post-open hooks at 1675846447731 (+4 ms)Region opened successfully at 1675846447732 (+1 ms) 2023-02-08 08:54:07,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659., pid=17, masterSystemTime=1675846447596 2023-02-08 08:54:07,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:07,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:07,737 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:07,737 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:07,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 67cb0b0aee3415fbb18966a88c15b8b5, NAME => 'testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.', STARTKEY => '', ENDKEY => '26'} 2023-02-08 08:54:07,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0c013ea68aeb4bc9d7e5174c5e923659, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:07,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:07,741 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447741"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846447741"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846447741"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846447741"}]},"ts":"1675846447741"} 2023-02-08 08:54:07,741 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1912): 0c013ea68aeb4bc9d7e5174c5e923659/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:07,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,741 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:07,742 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0c013ea68aeb4bc9d7e5174c5e923659/cf1 in testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,742 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1912): 0c013ea68aeb4bc9d7e5174c5e923659/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:07,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,743 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0c013ea68aeb4bc9d7e5174c5e923659/cf2 in testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:07,743 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/.tmp, totalSize=34.9 K 2023-02-08 08:54:07,745 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,745 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846445052 2023-02-08 08:54:07,746 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/.tmp, totalSize=34.9 K 2023-02-08 08:54:07,747 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846445422 2023-02-08 08:54:07,747 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846445052 2023-02-08 08:54:07,748 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1 2023-02-08 08:54:07,748 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1 2023-02-08 08:54:07,748 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 67cb0b0aee3415fbb18966a88c15b8b5 columnFamilyName cf1 2023-02-08 08:54:07,749 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846445684 2023-02-08 08:54:07,749 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846445422 2023-02-08 08:54:07,755 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting 22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846445684 2023-02-08 08:54:07,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-08 08:54:07,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 0c013ea68aeb4bc9d7e5174c5e923659, server=jenkins-hbase4.apache.org,39055,1675846441504 in 302 msec 2023-02-08 08:54:07,756 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846445924 2023-02-08 08:54:07,758 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846446154 2023-02-08 08:54:07,758 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846445924 2023-02-08 08:54:07,759 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846446393 2023-02-08 08:54:07,759 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846446154 2023-02-08 08:54:07,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, ASSIGN in 469 msec 2023-02-08 08:54:07,760 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846446393 2023-02-08 08:54:07,761 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846446613 2023-02-08 08:54:07,763 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846446613 2023-02-08 08:54:07,782 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-bottom 2023-02-08 08:54:07,790 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-bottom 2023-02-08 08:54:07,796 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c013ea68aeb4bc9d7e5174c5e923659#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:07,801 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c013ea68aeb4bc9d7e5174c5e923659#cf2#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:07,803 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-bottom 2023-02-08 08:54:07,811 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-bottom 2023-02-08 08:54:07,812 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(310): Store=67cb0b0aee3415fbb18966a88c15b8b5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:07,812 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,813 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2 2023-02-08 08:54:07,813 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2 2023-02-08 08:54:07,814 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 67cb0b0aee3415fbb18966a88c15b8b5 columnFamilyName cf2 2023-02-08 08:54:07,830 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/.tmp/cf1/2ceb38dbb743499faa661364247efe5c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/2ceb38dbb743499faa661364247efe5c 2023-02-08 08:54:07,832 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/.tmp/cf2/f8de1c4398374ca4b0d8376f28672907 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f8de1c4398374ca4b0d8376f28672907 2023-02-08 08:54:07,834 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-bottom 2023-02-08 08:54:07,849 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-bottom 2023-02-08 08:54:07,849 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-bottom 2023-02-08 08:54:07,853 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c013ea68aeb4bc9d7e5174c5e923659/cf1 of 0c013ea68aeb4bc9d7e5174c5e923659 into 2ceb38dbb743499faa661364247efe5c(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:07,853 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c013ea68aeb4bc9d7e5174c5e923659/cf2 of 0c013ea68aeb4bc9d7e5174c5e923659 into f8de1c4398374ca4b0d8376f28672907(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:07,856 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0c013ea68aeb4bc9d7e5174c5e923659: 2023-02-08 08:54:07,856 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0c013ea68aeb4bc9d7e5174c5e923659: 2023-02-08 08:54:07,856 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659., storeName=0c013ea68aeb4bc9d7e5174c5e923659/cf2, priority=9, startTime=1675846447737; duration=0sec 2023-02-08 08:54:07,856 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659., storeName=0c013ea68aeb4bc9d7e5174c5e923659/cf1, priority=9, startTime=1675846447735; duration=0sec 2023-02-08 08:54:07,857 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:07,857 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:07,858 DEBUG [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-bottom 2023-02-08 08:54:07,858 INFO [StoreOpener-67cb0b0aee3415fbb18966a88c15b8b5-1] regionserver.HStore(310): Store=67cb0b0aee3415fbb18966a88c15b8b5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:07,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:07,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:07,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 67cb0b0aee3415fbb18966a88c15b8b5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71419478, jitterRate=0.06423315405845642}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:07,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 67cb0b0aee3415fbb18966a88c15b8b5: Running coprocessor pre-open hook at 1675846447743Writing region info on filesystem at 1675846447743Initializing all the Stores at 1675846447744 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846447744Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846447744Cleaning up temporary data from old regions at 1675846447862 (+118 ms)Cleaning up detritus from prior splits at 1675846447862Running coprocessor post-open hooks at 1675846447866 (+4 ms)Region opened successfully at 1675846447866 2023-02-08 08:54:07,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5., pid=18, masterSystemTime=1675846447596 2023-02-08 08:54:07,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:07,868 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:54:07,868 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:54:07,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:07,868 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,868 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,868 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1912): 67cb0b0aee3415fbb18966a88c15b8b5/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:07,868 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1912): 67cb0b0aee3415fbb18966a88c15b8b5/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:07,868 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 67cb0b0aee3415fbb18966a88c15b8b5/cf1 in testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,868 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 67cb0b0aee3415fbb18966a88c15b8b5/cf2 in testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,869 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/.tmp, totalSize=19.9 K 2023-02-08 08:54:07,869 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/.tmp, totalSize=19.9 K 2023-02-08 08:54:07,869 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846445052 2023-02-08 08:54:07,870 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846445052 2023-02-08 08:54:07,870 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846445422 2023-02-08 08:54:07,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:07,871 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846445422 2023-02-08 08:54:07,871 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846445684 2023-02-08 08:54:07,872 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=67cb0b0aee3415fbb18966a88c15b8b5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:07,872 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting 22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846445684 2023-02-08 08:54:07,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846447872"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846447872"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846447872"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846447872"}]},"ts":"1675846447872"} 2023-02-08 08:54:07,873 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846445924 2023-02-08 08:54:07,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-08 08:54:07,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 67cb0b0aee3415fbb18966a88c15b8b5, server=jenkins-hbase4.apache.org,39055,1675846441504 in 425 msec 2023-02-08 08:54:07,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-08 08:54:07,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, ASSIGN in 592 msec 2023-02-08 08:54:07,883 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846445924 2023-02-08 08:54:07,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 in 1.2480 sec 2023-02-08 08:54:07,888 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] throttle.PressureAwareThroughputController(145): 67cb0b0aee3415fbb18966a88c15b8b5#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:07,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39055] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40188 deadline: 1675846507893, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b. is not online on jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:07,895 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] throttle.PressureAwareThroughputController(145): 67cb0b0aee3415fbb18966a88c15b8b5#cf1#compaction#17 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:07,917 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/.tmp/cf2/3907da8beee0465b901d7220f59a4edc as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/3907da8beee0465b901d7220f59a4edc 2023-02-08 08:54:07,923 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/.tmp/cf1/3f0adb9781de484498d0e2d6c0754fcc as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/3f0adb9781de484498d0e2d6c0754fcc 2023-02-08 08:54:07,923 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 67cb0b0aee3415fbb18966a88c15b8b5/cf2 of 67cb0b0aee3415fbb18966a88c15b8b5 into 3907da8beee0465b901d7220f59a4edc(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:07,923 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 67cb0b0aee3415fbb18966a88c15b8b5: 2023-02-08 08:54:07,923 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5., storeName=67cb0b0aee3415fbb18966a88c15b8b5/cf2, priority=12, startTime=1675846447868; duration=0sec 2023-02-08 08:54:07,924 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:07,929 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 67cb0b0aee3415fbb18966a88c15b8b5/cf1 of 67cb0b0aee3415fbb18966a88c15b8b5 into 3f0adb9781de484498d0e2d6c0754fcc(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:07,930 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 67cb0b0aee3415fbb18966a88c15b8b5: 2023-02-08 08:54:07,930 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5., storeName=67cb0b0aee3415fbb18966a88c15b8b5/cf1, priority=12, startTime=1675846447867; duration=0sec 2023-02-08 08:54:07,930 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:08,029 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 08:54:08,040 INFO [Listener at localhost/33105] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 08:54:08,044 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 08:54:08,082 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:08,083 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,083 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,091 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,091 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,092 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846448107"}]},"ts":"1675846448107"} 2023-02-08 08:54:08,109 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-08 08:54:08,111 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,111 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,112 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-08 08:54:08,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:54:08,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, UNASSIGN}] 2023-02-08 08:54:08,115 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, UNASSIGN 2023-02-08 08:54:08,116 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=0c013ea68aeb4bc9d7e5174c5e923659, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:08,116 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846448116"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846448116"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846448116"}]},"ts":"1675846448116"} 2023-02-08 08:54:08,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 0c013ea68aeb4bc9d7e5174c5e923659, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:08,121 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, UNASSIGN 2023-02-08 08:54:08,122 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=67cb0b0aee3415fbb18966a88c15b8b5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:08,122 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846448122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846448122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846448122"}]},"ts":"1675846448122"} 2023-02-08 08:54:08,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 67cb0b0aee3415fbb18966a88c15b8b5, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:08,216 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,216 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:54:08,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:08,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0c013ea68aeb4bc9d7e5174c5e923659, disabling compactions & flushes 2023-02-08 08:54:08,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:08,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:08,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. after waiting 0 ms 2023-02-08 08:54:08,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:08,282 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4] to archive 2023-02-08 08:54:08,283 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:08,288 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,290 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,292 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,293 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,295 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-d91ca415e785443fa9c6c7b018793d7b 2023-02-08 08:54:08,297 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-6a5771cd22c74e9e93e4ff5568b229ca 2023-02-08 08:54:08,298 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-58afdbaf25874a1d93931202a3da6cb4 2023-02-08 08:54:08,319 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11] to archive 2023-02-08 08:54:08,320 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:08,323 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,326 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,328 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,330 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,332 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b89b2c5ef81742be9737e109daea622a 2023-02-08 08:54:08,333 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-3fecc7b0d29a401eb5aab7626eae0285 2023-02-08 08:54:08,335 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/testSplitTableRegionUnevenDaughter=366926f8e963611bc9739ee35ac1108b-b63fb74e1c8c4b3fb2dfbcc1831a5c11 2023-02-08 08:54:08,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:54:08,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:08,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659. 2023-02-08 08:54:08,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0c013ea68aeb4bc9d7e5174c5e923659: Waiting for close lock at 1675846448270Running coprocessor pre-close hooks at 1675846448270Disabling compacts and flushes for region at 1675846448270Disabling writes for close at 1675846448271 (+1 ms)Writing region close event to WAL at 1675846448336 (+65 ms)Running coprocessor post-close hooks at 1675846448340 (+4 ms)Closed at 1675846448340 2023-02-08 08:54:08,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:08,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:08,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 67cb0b0aee3415fbb18966a88c15b8b5, disabling compactions & flushes 2023-02-08 08:54:08,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:08,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:08,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. after waiting 0 ms 2023-02-08 08:54:08,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:08,343 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=0c013ea68aeb4bc9d7e5174c5e923659, regionState=CLOSED 2023-02-08 08:54:08,343 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846448343"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846448343"}]},"ts":"1675846448343"} 2023-02-08 08:54:08,344 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28-bottom] to archive 2023-02-08 08:54:08,345 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:08,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-08 08:54:08,349 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/acba32b91bfc465ea90a39bbde72f925.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 0c013ea68aeb4bc9d7e5174c5e923659, server=jenkins-hbase4.apache.org,39055,1675846441504 in 228 msec 2023-02-08 08:54:08,351 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e26107add432444bb0ad121abe2dc374.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0c013ea68aeb4bc9d7e5174c5e923659, UNASSIGN in 235 msec 2023-02-08 08:54:08,353 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/22c481f5e9994ce28dfe389e7411458e.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,355 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/e6b0a964acba4a5cb897762803aeba28.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,357 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db-bottom] to archive 2023-02-08 08:54:08,357 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:08,359 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/defd7019e39c4ac2b799f252daa3440e.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,361 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/f22d695fbd5146a1ad0ee800b22e86db.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,362 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/a9a297fc793d4d45826bb1c7a53660dd.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,364 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/e37b1f7e31f24a629f47f03e39ed69db.366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:54:08,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:08,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5. 2023-02-08 08:54:08,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 67cb0b0aee3415fbb18966a88c15b8b5: Waiting for close lock at 1675846448342Running coprocessor pre-close hooks at 1675846448342Disabling compacts and flushes for region at 1675846448342Disabling writes for close at 1675846448343 (+1 ms)Writing region close event to WAL at 1675846448365 (+22 ms)Running coprocessor post-close hooks at 1675846448369 (+4 ms)Closed at 1675846448370 (+1 ms) 2023-02-08 08:54:08,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:08,373 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=67cb0b0aee3415fbb18966a88c15b8b5, regionState=CLOSED 2023-02-08 08:54:08,373 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846448373"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846448373"}]},"ts":"1675846448373"} 2023-02-08 08:54:08,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-08 08:54:08,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 67cb0b0aee3415fbb18966a88c15b8b5, server=jenkins-hbase4.apache.org,39055,1675846441504 in 252 msec 2023-02-08 08:54:08,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-08 08:54:08,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=67cb0b0aee3415fbb18966a88c15b8b5, UNASSIGN in 264 msec 2023-02-08 08:54:08,382 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846448382"}]},"ts":"1675846448382"} 2023-02-08 08:54:08,384 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-08 08:54:08,387 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-08 08:54:08,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 292 msec 2023-02-08 08:54:08,418 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,419 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:54:08,423 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-08 08:54:08,423 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,424 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,453 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,454 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,454 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 08:54:08,464 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,465 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:08,465 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:08,470 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/recovered.edits] 2023-02-08 08:54:08,470 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/recovered.edits] 2023-02-08 08:54:08,470 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/recovered.edits] 2023-02-08 08:54:08,477 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/3f0adb9781de484498d0e2d6c0754fcc to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf1/3f0adb9781de484498d0e2d6c0754fcc 2023-02-08 08:54:08,477 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/2ceb38dbb743499faa661364247efe5c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf1/2ceb38dbb743499faa661364247efe5c 2023-02-08 08:54:08,478 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/22c481f5e9994ce28dfe389e7411458e 2023-02-08 08:54:08,481 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/58afdbaf25874a1d93931202a3da6cb4 2023-02-08 08:54:08,483 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/3907da8beee0465b901d7220f59a4edc to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/cf2/3907da8beee0465b901d7220f59a4edc 2023-02-08 08:54:08,483 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/6a5771cd22c74e9e93e4ff5568b229ca 2023-02-08 08:54:08,483 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f8de1c4398374ca4b0d8376f28672907 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/cf2/f8de1c4398374ca4b0d8376f28672907 2023-02-08 08:54:08,485 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/acba32b91bfc465ea90a39bbde72f925 2023-02-08 08:54:08,489 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/d91ca415e785443fa9c6c7b018793d7b 2023-02-08 08:54:08,489 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5/recovered.edits/90.seqid 2023-02-08 08:54:08,490 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659/recovered.edits/90.seqid 2023-02-08 08:54:08,490 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/67cb0b0aee3415fbb18966a88c15b8b5 2023-02-08 08:54:08,491 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:08,492 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e26107add432444bb0ad121abe2dc374 2023-02-08 08:54:08,494 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf1/e6b0a964acba4a5cb897762803aeba28 2023-02-08 08:54:08,497 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/3fecc7b0d29a401eb5aab7626eae0285 2023-02-08 08:54:08,499 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/a9a297fc793d4d45826bb1c7a53660dd 2023-02-08 08:54:08,501 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b63fb74e1c8c4b3fb2dfbcc1831a5c11 2023-02-08 08:54:08,503 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/b89b2c5ef81742be9737e109daea622a 2023-02-08 08:54:08,504 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/defd7019e39c4ac2b799f252daa3440e 2023-02-08 08:54:08,506 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/e37b1f7e31f24a629f47f03e39ed69db 2023-02-08 08:54:08,508 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/cf2/f22d695fbd5146a1ad0ee800b22e86db 2023-02-08 08:54:08,511 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b/recovered.edits/85.seqid 2023-02-08 08:54:08,515 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionUnevenDaughter/366926f8e963611bc9739ee35ac1108b 2023-02-08 08:54:08,515 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 08:54:08,518 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,543 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-08 08:54:08,556 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,557 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 08:54:08,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-08 08:54:08,586 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,587 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-08 08:54:08,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846448587"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:08,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846448587"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:08,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846448587"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:08,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:54:08,590 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 366926f8e963611bc9739ee35ac1108b, NAME => 'testSplitTableRegionUnevenDaughter,,1675846444541.366926f8e963611bc9739ee35ac1108b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 67cb0b0aee3415fbb18966a88c15b8b5, NAME => 'testSplitTableRegionUnevenDaughter,,1675846446634.67cb0b0aee3415fbb18966a88c15b8b5.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 0c013ea68aeb4bc9d7e5174c5e923659, NAME => 'testSplitTableRegionUnevenDaughter,26,1675846446634.0c013ea68aeb4bc9d7e5174c5e923659.', STARTKEY => '26', ENDKEY => ''}] 2023-02-08 08:54:08,590 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-08 08:54:08,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846448590"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:08,593 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-08 08:54:08,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:08,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 154 msec 2023-02-08 08:54:08,758 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,758 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 08:54:08,760 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-08 08:54:08,776 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=393 (was 361) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49264 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5d1b4a5f-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:45662 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49494 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5d1b4a5f-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49528 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49492 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:56620 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49170 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:56394 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:45340 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:56602 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49154 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:39055-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49306 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:45674 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:45402 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:56618 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_447261374_15 at /127.0.0.1:49512 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=865 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=288 (was 269) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4489 (was 4629) 2023-02-08 08:54:08,791 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=393, OpenFileDescriptor=865, MaxFileDescriptor=60000, SystemLoadAverage=288, ProcessCount=168, AvailableMemoryMB=4488 2023-02-08 08:54:08,791 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:08,791 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,791 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,796 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:08,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:08,798 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:08,800 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:08,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:08,803 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:08,804 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf empty. 2023-02-08 08:54:08,804 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:08,804 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 08:54:08,816 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:08,817 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4c3aeb6b729e6681efb5e73977d7b2cf, NAME => 'testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 4c3aeb6b729e6681efb5e73977d7b2cf, disabling compactions & flushes 2023-02-08 08:54:08,827 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. after waiting 0 ms 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:08,827 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:08,827 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 4c3aeb6b729e6681efb5e73977d7b2cf: Waiting for close lock at 1675846448827Disabling compacts and flushes for region at 1675846448827Disabling writes for close at 1675846448827Writing region close event to WAL at 1675846448827Closed at 1675846448827 2023-02-08 08:54:08,830 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:08,831 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846448831"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846448831"}]},"ts":"1675846448831"} 2023-02-08 08:54:08,833 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:08,834 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:08,834 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846448834"}]},"ts":"1675846448834"} 2023-02-08 08:54:08,836 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-08 08:54:08,839 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:08,840 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:08,841 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:08,841 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:08,841 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:08,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, ASSIGN}] 2023-02-08 08:54:08,843 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, ASSIGN 2023-02-08 08:54:08,843 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:08,940 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:08,994 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:08,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:08,997 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846448995"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846448995"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846448995"}]},"ts":"1675846448995"} 2023-02-08 08:54:08,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:09,007 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-08 08:54:09,008 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-08 08:54:09,152 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,153 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:54:09,156 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49616, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:54:09,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4c3aeb6b729e6681efb5e73977d7b2cf, NAME => 'testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:09,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:09,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:09,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,162 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,164 DEBUG [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1 2023-02-08 08:54:09,164 DEBUG [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1 2023-02-08 08:54:09,165 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4c3aeb6b729e6681efb5e73977d7b2cf columnFamilyName cf1 2023-02-08 08:54:09,166 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] regionserver.HStore(310): Store=4c3aeb6b729e6681efb5e73977d7b2cf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:09,166 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,167 DEBUG [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2 2023-02-08 08:54:09,168 DEBUG [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2 2023-02-08 08:54:09,169 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4c3aeb6b729e6681efb5e73977d7b2cf columnFamilyName cf2 2023-02-08 08:54:09,170 INFO [StoreOpener-4c3aeb6b729e6681efb5e73977d7b2cf-1] regionserver.HStore(310): Store=4c3aeb6b729e6681efb5e73977d7b2cf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:09,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:09,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:09,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:09,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4c3aeb6b729e6681efb5e73977d7b2cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72174893, jitterRate=0.07548971474170685}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4c3aeb6b729e6681efb5e73977d7b2cf: Running coprocessor pre-open hook at 1675846449161Writing region info on filesystem at 1675846449161Initializing all the Stores at 1675846449162 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846449162Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846449162Cleaning up temporary data from old regions at 1675846449172 (+10 ms)Cleaning up detritus from prior splits at 1675846449173 (+1 ms)Running coprocessor post-open hooks at 1675846449181 (+8 ms)Region opened successfully at 1675846449181 2023-02-08 08:54:09,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf., pid=27, masterSystemTime=1675846449152 2023-02-08 08:54:09,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,187 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846449187"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846449187"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846449187"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846449187"}]},"ts":"1675846449187"} 2023-02-08 08:54:09,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-08 08:54:09,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 in 190 msec 2023-02-08 08:54:09,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-08 08:54:09,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, ASSIGN in 351 msec 2023-02-08 08:54:09,196 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:09,197 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846449196"}]},"ts":"1675846449196"} 2023-02-08 08:54:09,198 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-08 08:54:09,201 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:09,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 404 msec 2023-02-08 08:54:09,302 DEBUG [Listener at localhost/33105] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:09,308 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49624, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:09,317 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,317 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:09,320 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:09,320 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,321 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:09,322 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:09,322 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,322 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,324 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,329 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,329 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,329 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,329 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,329 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,329 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,329 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,330 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,329 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,331 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,331 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:09,331 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,332 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:54:09,333 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,334 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,334 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,346 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/19c94d08c0894d929430093894dc8665 2023-02-08 08:54:09,360 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/37128f1bad744f44be02e014491f2f96 2023-02-08 08:54:09,365 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/19c94d08c0894d929430093894dc8665 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 2023-02-08 08:54:09,370 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:09,371 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/37128f1bad744f44be02e014491f2f96 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 2023-02-08 08:54:09,376 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:09,377 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 45ms, sequenceid=6, compaction requested=false 2023-02-08 08:54:09,377 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-08 08:54:09,378 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:09,378 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:09,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:09,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,380 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,380 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,380 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,380 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:09,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35f20109[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:09,382 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:09,382 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,385 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,385 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,385 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:09,386 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:09,386 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,388 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,388 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,390 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,390 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,390 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:09,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:09,394 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:09,394 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:09,394 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:09,394 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:09,394 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,398 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,398 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,399 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,401 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,401 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,401 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:09,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:09,405 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,404 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:09,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:09,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,406 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,407 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,407 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,506 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:09,506 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,506 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:09,525 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,525 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:09,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:09,528 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:09,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,529 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:09,529 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:09,529 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,529 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,530 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,531 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,531 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,532 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,534 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,534 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,534 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,534 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,534 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,535 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,536 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,536 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,536 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,537 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,537 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,537 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:09,537 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:09,538 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,539 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,539 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,548 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/6fde51fd1aca4f8dab5aa05553a96d18 2023-02-08 08:54:09,564 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/7233ec322e644286b4c67bc51b7f7cc7 2023-02-08 08:54:09,569 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/6fde51fd1aca4f8dab5aa05553a96d18 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 2023-02-08 08:54:09,573 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:09,574 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/7233ec322e644286b4c67bc51b7f7cc7 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 2023-02-08 08:54:09,579 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:09,580 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 43ms, sequenceid=19, compaction requested=false 2023-02-08 08:54:09,580 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:09,580 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,582 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,582 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:09,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78efd8b8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:09,584 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:09,584 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,586 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,586 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:09,586 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:09,590 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,590 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,590 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,590 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,591 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,590 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,592 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,592 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,592 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,592 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:09,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:09,596 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:09,596 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:09,596 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:09,596 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:09,596 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,598 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,599 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,599 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,607 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:09,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,607 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:09,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,710 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:09,710 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,710 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:09,728 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,728 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:09,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:09,731 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:09,731 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,732 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:09,732 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:09,732 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,732 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,734 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,734 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,734 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,735 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,735 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,737 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,737 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,737 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,737 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,738 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,737 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,738 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,738 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,739 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,739 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,740 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:09,740 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:09,741 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,742 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,742 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,742 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,742 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,742 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,752 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/9929c7ad984b4653877770308e0d27a4 2023-02-08 08:54:09,767 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/e4996ac184ea43f9ba5b52b4c96a650e 2023-02-08 08:54:09,772 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/9929c7ad984b4653877770308e0d27a4 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 2023-02-08 08:54:09,778 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:09,779 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/e4996ac184ea43f9ba5b52b4c96a650e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e 2023-02-08 08:54:09,784 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:09,785 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 45ms, sequenceid=32, compaction requested=true 2023-02-08 08:54:09,785 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:09,786 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,787 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,787 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,788 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,788 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53a58259[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:09,790 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:09,790 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,791 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,791 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,791 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,791 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:09,792 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,793 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,793 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:09,794 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,795 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:09,797 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:09,798 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:09,798 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:09,798 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:09,798 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,800 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,801 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,801 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:09,803 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,803 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,804 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:09,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:09,806 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,806 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,807 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:09,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,807 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:09,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:09,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,808 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,808 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,808 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:09,808 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,808 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,908 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:09,908 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,908 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:09,927 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,927 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:09,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:09,930 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:09,930 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:09,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:09,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,933 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,933 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,933 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,933 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,934 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:09,934 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,936 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:09,936 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,936 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,936 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:09,938 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,937 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,939 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:09,938 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:09,940 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,940 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:09,940 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,940 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:09,940 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:09,943 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,944 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,944 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,944 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,944 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:09,944 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,356 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/f1b73fb138ca4985810d3feea95d63c2 2023-02-08 08:54:10,371 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/815c0408cd2e414aa8b2cbbb84133a26 2023-02-08 08:54:10,376 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/f1b73fb138ca4985810d3feea95d63c2 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 2023-02-08 08:54:10,381 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:10,382 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/815c0408cd2e414aa8b2cbbb84133a26 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 2023-02-08 08:54:10,387 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:10,388 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 448ms, sequenceid=45, compaction requested=true 2023-02-08 08:54:10,388 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:10,388 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:10,388 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:10,388 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,390 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,390 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53c48599[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:10,393 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:10,393 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,398 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:10,398 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:10,400 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,400 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,400 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,401 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,401 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:10,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:10,404 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:10,404 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:10,404 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:10,404 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:10,404 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,405 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,406 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,406 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,406 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,408 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,409 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,409 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,413 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:10,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:10,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,413 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,414 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:10,414 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,414 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:10,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,414 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,415 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,445 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 08:54:10,446 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-08 08:54:10,446 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:10,446 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-08 08:54:10,515 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:10,515 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,515 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:10,533 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,533 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:10,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:10,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:10,536 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:10,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:10,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:10,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,539 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,540 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,540 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,540 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,540 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,540 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:10,540 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:10,540 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:10,541 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:10,541 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,558 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/d6d6229f1845417cb8191b47f1edaf81 2023-02-08 08:54:10,575 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/e5c3e74c15b24592a24ff53722f3fb2c 2023-02-08 08:54:10,582 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/d6d6229f1845417cb8191b47f1edaf81 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 2023-02-08 08:54:10,587 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:10,588 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/e5c3e74c15b24592a24ff53722f3fb2c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c 2023-02-08 08:54:10,593 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:10,594 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 53ms, sequenceid=58, compaction requested=true 2023-02-08 08:54:10,595 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:10,595 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:10,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:10,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:10,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,596 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,596 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,598 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,598 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,599 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:10,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@769b3ca1[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:10,601 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:10,601 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,603 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:10,602 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,603 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:10,604 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,604 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:10,608 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:10,608 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:10,608 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:10,608 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:10,608 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:10,609 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,609 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,610 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,610 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,611 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,611 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,613 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,613 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,613 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:10,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:10,619 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,619 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,620 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:10,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,621 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,721 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:10,721 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,721 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:10,742 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,742 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:10,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:10,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 08:54:10,745 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 08:54:10,745 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,745 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 08:54:10,746 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 08:54:10,746 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,746 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,747 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,747 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,747 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,747 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,748 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,748 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,750 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:10,750 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. started... 2023-02-08 08:54:10,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,750 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:10,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 08:54:10,750 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 08:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 08:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,752 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,753 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,764 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/684935ab2c704e78ada3ff4b86eb9f65 2023-02-08 08:54:10,778 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/72a864cd3d1743c2967fc73e4cc9cce5 2023-02-08 08:54:10,783 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/684935ab2c704e78ada3ff4b86eb9f65 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 2023-02-08 08:54:10,788 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:10,789 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/72a864cd3d1743c2967fc73e4cc9cce5 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 2023-02-08 08:54:10,793 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:10,794 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 44ms, sequenceid=71, compaction requested=true 2023-02-08 08:54:10,795 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4c3aeb6b729e6681efb5e73977d7b2cf: 2023-02-08 08:54:10,795 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:10,795 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:10,795 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:10,795 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,795 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 08:54:10,795 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 08:54:10,797 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,797 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,798 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,798 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 08:54:10,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70b710d2[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:10,799 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 08:54:10,799 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,801 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,801 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:10,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:10,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,803 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,803 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 08:54:10,804 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,804 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,804 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:10,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 08:54:10,807 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 08:54:10,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:10,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:10,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 08:54:10,807 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,810 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,810 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,811 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,812 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,813 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,813 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,813 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 08:54:10,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 08:54:10,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:10,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,816 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,817 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:10,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:10,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 08:54:10,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 08:54:10,918 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:10,918 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,918 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:11,182 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 08:54:11,183 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:11,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,186 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:11,187 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:11,187 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:11,435 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,436 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:11,436 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:11,436 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:11,438 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-08 08:54:11,439 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:11,439 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:11,439 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:11,439 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:11,440 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:11,443 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 08:54:11,444 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:11,445 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:11,445 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:11,446 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:11,446 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,447 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:11,447 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,447 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,448 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:11,448 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,448 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:11,448 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,449 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,449 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,449 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 08:54:11,450 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:11,450 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:11,451 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846481451 2023-02-08 08:54:11,452 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,452 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:11,456 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:11,456 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:11,456 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:11,456 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846511456 2023-02-08 08:54:11,456 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846571456 2023-02-08 08:54:11,456 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:11,457 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:11,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:11,459 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:11,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:11,707 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:11,707 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:11,708 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:11,708 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:11,710 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 08:54:11,710 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:11,711 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:11,711 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:11,711 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:11,711 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,712 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:11,712 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,712 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,713 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:11,713 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,713 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:11,713 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,713 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,713 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,714 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:11,714 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:11,715 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:11,715 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846481715 2023-02-08 08:54:11,716 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:11,716 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,719 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:11,720 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:11,720 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:11,720 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846511720 2023-02-08 08:54:11,720 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846571720 2023-02-08 08:54:11,720 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:11,721 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:11,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN}] 2023-02-08 08:54:11,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:11,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:11,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:11,971 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:11,971 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:11,972 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:11,972 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:11,974 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 08:54:11,974 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:11,974 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:11,975 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:11,975 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:11,976 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,976 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:11,976 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,976 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:11,977 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:11,977 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,977 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:11,977 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,978 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,978 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,978 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:11,978 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:11,979 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:11,979 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846481979 2023-02-08 08:54:11,981 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:11,981 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:11,983 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:11,983 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:11,984 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:11,984 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846511984 2023-02-08 08:54:11,984 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846571984 2023-02-08 08:54:11,984 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:11,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN}] 2023-02-08 08:54:11,984 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:11,986 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:11,987 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:11,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846451987"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846451987"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846451987"}]},"ts":"1675846451987"} 2023-02-08 08:54:11,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:11,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:11,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:11,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:11,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:12,234 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,234 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:12,234 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:12,234 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:12,235 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:12,235 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:12,235 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:12,236 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:12,236 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:12,236 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:12,238 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:12,238 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:12,238 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:12,239 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:12,239 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:12,239 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:12,240 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,240 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:12,240 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,240 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,241 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:12,242 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,242 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,242 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,242 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,242 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,242 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,242 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,242 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,242 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:12,243 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:12,243 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:12,243 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf to restore RIT 2023-02-08 08:54:12,244 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846482244 2023-02-08 08:54:12,245 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,245 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:12,247 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:12,248 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:12,248 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:12,248 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846512248 2023-02-08 08:54:12,248 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846572248 2023-02-08 08:54:12,248 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:12,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846452248"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846452248"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846452248"}]},"ts":"1675846452248"} 2023-02-08 08:54:12,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:12,250 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:12,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:12,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4c3aeb6b729e6681efb5e73977d7b2cf, disabling compactions & flushes 2023-02-08 08:54:12,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:12,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:12,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. after waiting 0 ms 2023-02-08 08:54:12,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:12,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4c3aeb6b729e6681efb5e73977d7b2cf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:54:12,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/da4cd6f12ee34be0a61c986fb38dfc5a 2023-02-08 08:54:12,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/a6ea560f0e4049b0afe2e4a910503ba0 2023-02-08 08:54:12,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf1/da4cd6f12ee34be0a61c986fb38dfc5a as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a 2023-02-08 08:54:12,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:12,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/.tmp/cf2/a6ea560f0e4049b0afe2e4a910503ba0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 2023-02-08 08:54:12,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:12,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 43ms, sequenceid=82, compaction requested=true 2023-02-08 08:54:12,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:54:12,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:12,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. 2023-02-08 08:54:12,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4c3aeb6b729e6681efb5e73977d7b2cf: Waiting for close lock at 1675846452404Running coprocessor pre-close hooks at 1675846452404Disabling compacts and flushes for region at 1675846452404Disabling writes for close at 1675846452404Obtaining lock to block concurrent updates at 1675846452404Preparing flush snapshotting stores in 4c3aeb6b729e6681efb5e73977d7b2cf at 1675846452404Finished memstore snapshotting testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846452404Flushing stores of testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. at 1675846452405 (+1 ms)Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf1: creating writer at 1675846452405Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf1: appending metadata at 1675846452407 (+2 ms)Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf1: closing flushed file at 1675846452407Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf2: creating writer at 1675846452421 (+14 ms)Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf2: appending metadata at 1675846452423 (+2 ms)Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf2: closing flushed file at 1675846452423Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf1: reopening flushed file at 1675846452437 (+14 ms)Flushing 4c3aeb6b729e6681efb5e73977d7b2cf/cf2: reopening flushed file at 1675846452442 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4c3aeb6b729e6681efb5e73977d7b2cf in 43ms, sequenceid=82, compaction requested=true at 1675846452447 (+5 ms)Writing region close event to WAL at 1675846452464 (+17 ms)Running coprocessor post-close hooks at 1675846452468 (+4 ms)Closed at 1675846452468 2023-02-08 08:54:12,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:12,471 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED 2023-02-08 08:54:12,471 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846452471"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846452471"}]},"ts":"1675846452471"} 2023-02-08 08:54:12,473 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:12,473 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:12,474 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:12,498 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,498 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:12,498 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:12,498 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:12,499 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:12,499 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:12,499 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:12,499 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:12,499 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:12,499 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:12,501 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:12,501 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 08:54:12,501 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 08:54:12,501 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:12,502 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:12,502 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:12,502 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:12,502 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,503 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:12,503 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,503 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,503 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:12,504 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,504 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,504 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,504 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,504 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,505 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,505 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,505 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,505 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,505 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,505 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:12,505 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:12,506 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:12,506 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf to restore RIT 2023-02-08 08:54:12,506 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846482506 2023-02-08 08:54:12,506 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:12,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,509 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:12,510 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:12,510 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:12,510 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846512510 2023-02-08 08:54:12,510 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846572510 2023-02-08 08:54:12,510 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:12,510 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846452510"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846452510"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846452510"}]},"ts":"1675846452510"} 2023-02-08 08:54:12,512 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:12,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-08 08:54:12,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 in 262 msec 2023-02-08 08:54:12,516 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:12,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:12,516 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:12,760 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,761 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:12,761 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:12,761 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:12,762 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:12,762 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:12,763 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:12,763 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:12,763 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:12,763 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:12,765 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:12,765 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 08:54:12,765 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:12,772 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:12,773 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:12,773 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:12,773 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:12,774 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,774 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:12,774 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,775 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:12,775 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:12,776 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,776 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,776 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:12,776 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:12,777 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,777 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,777 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,777 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:12,777 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:12,777 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-08 08:54:12,778 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:12,778 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:12,779 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf to restore RIT 2023-02-08 08:54:12,779 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846482779 2023-02-08 08:54:12,779 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:12,780 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,782 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:12,783 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:12,783 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:12,783 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846512783 2023-02-08 08:54:12,783 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846572783 2023-02-08 08:54:12,783 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:12,783 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:12,785 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:12,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-08 08:54:12,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN in 801 msec 2023-02-08 08:54:12,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:12,792 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:12,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:12,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:13,033 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,033 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:13,033 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:13,033 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:13,034 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:13,034 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:13,034 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:13,034 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:13,034 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:13,035 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:13,036 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:13,036 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:13,036 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:13,036 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:13,037 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:13,037 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:13,037 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:13,037 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,038 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:13,038 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,038 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,038 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:13,039 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:13,040 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:13,040 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:13,041 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:13,041 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846483041 2023-02-08 08:54:13,042 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:13,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,045 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:13,045 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:13,045 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:13,045 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846513045 2023-02-08 08:54:13,045 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846573045 2023-02-08 08:54:13,045 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:13,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,046 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,051 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:13,060 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4c3aeb6b729e6681efb5e73977d7b2cf, threads=14 2023-02-08 08:54:13,061 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,061 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,061 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,062 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,062 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,062 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,063 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,063 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,063 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,063 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,063 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,064 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,064 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,064 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,174 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,182 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,185 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,186 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,205 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,205 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,205 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,206 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,207 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,208 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,210 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,212 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,222 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,224 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,224 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4c3aeb6b729e6681efb5e73977d7b2cf Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:13,261 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:13,261 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:13,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:13,296 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,296 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:13,296 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:13,296 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:13,297 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:13,297 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:13,297 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:13,297 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:13,297 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:13,298 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:13,299 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 08:54:13,299 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:13,299 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:13,299 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:13,300 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:13,300 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:13,300 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:13,301 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,301 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:13,301 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,302 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:13,302 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:13,303 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,303 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,303 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:13,303 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:13,303 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:13,304 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:13,305 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:13,307 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846483307 2023-02-08 08:54:13,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:13,309 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:13,311 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:13,311 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:13,312 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:13,312 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846513312 2023-02-08 08:54:13,312 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846573312 2023-02-08 08:54:13,312 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:13,321 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4c3aeb6b729e6681efb5e73977d7b2cf, threads=14 2023-02-08 08:54:13,322 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,322 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,324 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,324 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,324 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,328 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,328 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,328 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,332 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,332 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,332 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,333 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,334 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,334 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,496 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,512 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,513 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,528 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,528 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,541 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,542 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,544 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,544 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,556 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,556 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,567 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,568 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,843 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:13,965 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a for region: 4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:13,965 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4c3aeb6b729e6681efb5e73977d7b2cf Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:14,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:14,034 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:14,036 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:14,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:14,036 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:14,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:14,062 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,062 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:14,062 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:14,062 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:14,063 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:14,063 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:14,063 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:14,063 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:14,063 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:14,064 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:14,065 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 08:54:14,065 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:14,065 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:14,065 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:14,066 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:14,066 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:14,066 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:14,067 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,067 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:14,067 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,067 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,068 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:14,068 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,069 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:14,069 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,069 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:14,069 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,069 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,069 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,069 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:14,069 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:14,069 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:14,070 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:14,070 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:14,071 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846484071 2023-02-08 08:54:14,073 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,073 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:14,075 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:14,076 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:14,076 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:14,076 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846514076 2023-02-08 08:54:14,076 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846574076 2023-02-08 08:54:14,076 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:14,079 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:14,082 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:14,082 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:14,082 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:14,326 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:14,327 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:14,327 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:14,328 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:14,329 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 08:54:14,329 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:14,329 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:14,329 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:14,330 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:14,330 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:14,330 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:14,330 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,330 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:14,331 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,331 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:14,331 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:14,332 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,332 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,332 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4c3aeb6b729e6681efb5e73977d7b2cf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:14,332 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4c3aeb6b729e6681efb5e73977d7b2cf, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:14,332 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:14,333 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:14,333 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:14,334 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846484334 2023-02-08 08:54:14,335 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:14,335 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,337 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:14,337 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=4c3aeb6b729e6681efb5e73977d7b2cf, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:14,337 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:14,337 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846514337 2023-02-08 08:54:14,337 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846574337 2023-02-08 08:54:14,338 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:14,341 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846454341"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675846454341"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675846454341"}]},"ts":"1675846454341"} 2023-02-08 08:54:14,341 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454341"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846454341"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454341"}]},"ts":"1675846454341"} 2023-02-08 08:54:14,341 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454341"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846454341"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454341"}]},"ts":"1675846454341"} 2023-02-08 08:54:14,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, ASSIGN}] 2023-02-08 08:54:14,347 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, ASSIGN 2023-02-08 08:54:14,348 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:14,349 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, ASSIGN 2023-02-08 08:54:14,349 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:14,498 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:54:14,500 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4640d7a44b2f6cb5eae699f90c14ecc9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454500"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846454500"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846454500"}]},"ts":"1675846454500"} 2023-02-08 08:54:14,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 4640d7a44b2f6cb5eae699f90c14ecc9, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:14,505 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=df6b1a9175455ee27dbcef5fc4d3c6b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,505 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454505"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846454505"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846454505"}]},"ts":"1675846454505"} 2023-02-08 08:54:14,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure df6b1a9175455ee27dbcef5fc4d3c6b6, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:14,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => df6b1a9175455ee27dbcef5fc4d3c6b6, NAME => 'testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 08:54:14,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:14,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:14,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,660 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,661 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1 2023-02-08 08:54:14,661 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1 2023-02-08 08:54:14,662 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df6b1a9175455ee27dbcef5fc4d3c6b6 columnFamilyName cf1 2023-02-08 08:54:14,680 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-top 2023-02-08 08:54:14,685 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-top 2023-02-08 08:54:14,689 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-top 2023-02-08 08:54:14,694 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-top 2023-02-08 08:54:14,699 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-top 2023-02-08 08:54:14,703 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-top 2023-02-08 08:54:14,707 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-top 2023-02-08 08:54:14,708 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(310): Store=df6b1a9175455ee27dbcef5fc4d3c6b6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,708 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,709 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2 2023-02-08 08:54:14,709 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2 2023-02-08 08:54:14,709 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df6b1a9175455ee27dbcef5fc4d3c6b6 columnFamilyName cf2 2023-02-08 08:54:14,723 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-top 2023-02-08 08:54:14,728 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-top 2023-02-08 08:54:14,732 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-top 2023-02-08 08:54:14,736 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-top 2023-02-08 08:54:14,740 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-top 2023-02-08 08:54:14,745 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-top 2023-02-08 08:54:14,749 DEBUG [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-top 2023-02-08 08:54:14,749 INFO [StoreOpener-df6b1a9175455ee27dbcef5fc4d3c6b6-1] regionserver.HStore(310): Store=df6b1a9175455ee27dbcef5fc4d3c6b6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:14,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:14,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened df6b1a9175455ee27dbcef5fc4d3c6b6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63230048, jitterRate=-0.05779886245727539}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:14,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for df6b1a9175455ee27dbcef5fc4d3c6b6: Running coprocessor pre-open hook at 1675846454658Writing region info on filesystem at 1675846454658Initializing all the Stores at 1675846454659 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846454659Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846454660 (+1 ms)Cleaning up temporary data from old regions at 1675846454751 (+91 ms)Cleaning up detritus from prior splits at 1675846454752 (+1 ms)Running coprocessor post-open hooks at 1675846454754 (+2 ms)Region opened successfully at 1675846454754 2023-02-08 08:54:14,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6., pid=34, masterSystemTime=1675846454654 2023-02-08 08:54:14,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:14,757 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:14,757 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,757 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): df6b1a9175455ee27dbcef5fc4d3c6b6/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of df6b1a9175455ee27dbcef5fc4d3c6b6/cf1 in testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,757 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): df6b1a9175455ee27dbcef5fc4d3c6b6/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of df6b1a9175455ee27dbcef5fc4d3c6b6/cf2 in testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/.tmp, totalSize=34.9 K 2023-02-08 08:54:14,757 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/.tmp, totalSize=34.9 K 2023-02-08 08:54:14,758 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846449309 2023-02-08 08:54:14,758 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846449309 2023-02-08 08:54:14,759 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846449508 2023-02-08 08:54:14,759 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846449508 2023-02-08 08:54:14,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:14,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,759 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846449712 2023-02-08 08:54:14,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4640d7a44b2f6cb5eae699f90c14ecc9, NAME => 'testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 08:54:14,759 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=df6b1a9175455ee27dbcef5fc4d3c6b6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:14,759 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846449712 2023-02-08 08:54:14,760 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454759"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846454759"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846454759"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454759"}]},"ts":"1675846454759"} 2023-02-08 08:54:14,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:14,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,760 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846449910 2023-02-08 08:54:14,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,760 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846449910 2023-02-08 08:54:14,761 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846450517 2023-02-08 08:54:14,761 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846450517 2023-02-08 08:54:14,761 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846450723 2023-02-08 08:54:14,762 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,762 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846450723 2023-02-08 08:54:14,762 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846450920 2023-02-08 08:54:14,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-08 08:54:14,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure df6b1a9175455ee27dbcef5fc4d3c6b6, server=jenkins-hbase4.apache.org,44651,1675846441425 in 256 msec 2023-02-08 08:54:14,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, ASSIGN in 419 msec 2023-02-08 08:54:14,766 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846450920 2023-02-08 08:54:14,768 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1 2023-02-08 08:54:14,768 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1 2023-02-08 08:54:14,768 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4640d7a44b2f6cb5eae699f90c14ecc9 columnFamilyName cf1 2023-02-08 08:54:14,785 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): df6b1a9175455ee27dbcef5fc4d3c6b6#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:14,787 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): df6b1a9175455ee27dbcef5fc4d3c6b6#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:14,791 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-bottom 2023-02-08 08:54:14,795 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-bottom 2023-02-08 08:54:14,803 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-bottom 2023-02-08 08:54:14,807 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-bottom 2023-02-08 08:54:14,812 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-bottom 2023-02-08 08:54:14,821 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-bottom 2023-02-08 08:54:14,827 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/.tmp/cf1/db33cd3302fe4b60a86e367cde8dd31c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/db33cd3302fe4b60a86e367cde8dd31c 2023-02-08 08:54:14,827 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-bottom 2023-02-08 08:54:14,828 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(310): Store=4640d7a44b2f6cb5eae699f90c14ecc9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,828 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,829 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2 2023-02-08 08:54:14,829 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2 2023-02-08 08:54:14,830 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4640d7a44b2f6cb5eae699f90c14ecc9 columnFamilyName cf2 2023-02-08 08:54:14,831 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/.tmp/cf2/f941d7e3f54045b18928406200e8be16 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/f941d7e3f54045b18928406200e8be16 2023-02-08 08:54:14,835 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in df6b1a9175455ee27dbcef5fc4d3c6b6/cf1 of df6b1a9175455ee27dbcef5fc4d3c6b6 into db33cd3302fe4b60a86e367cde8dd31c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:14,835 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for df6b1a9175455ee27dbcef5fc4d3c6b6: 2023-02-08 08:54:14,835 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6., storeName=df6b1a9175455ee27dbcef5fc4d3c6b6/cf1, priority=9, startTime=1675846454755; duration=0sec 2023-02-08 08:54:14,835 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,842 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in df6b1a9175455ee27dbcef5fc4d3c6b6/cf2 of df6b1a9175455ee27dbcef5fc4d3c6b6 into f941d7e3f54045b18928406200e8be16(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:14,842 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for df6b1a9175455ee27dbcef5fc4d3c6b6: 2023-02-08 08:54:14,842 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6., storeName=df6b1a9175455ee27dbcef5fc4d3c6b6/cf2, priority=9, startTime=1675846454756; duration=0sec 2023-02-08 08:54:14,842 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,852 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-bottom 2023-02-08 08:54:14,856 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-bottom 2023-02-08 08:54:14,860 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-bottom 2023-02-08 08:54:14,864 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-bottom 2023-02-08 08:54:14,868 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-bottom 2023-02-08 08:54:14,872 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-bottom 2023-02-08 08:54:14,875 DEBUG [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-bottom 2023-02-08 08:54:14,875 INFO [StoreOpener-4640d7a44b2f6cb5eae699f90c14ecc9-1] regionserver.HStore(310): Store=4640d7a44b2f6cb5eae699f90c14ecc9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:14,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:14,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4640d7a44b2f6cb5eae699f90c14ecc9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66648673, jitterRate=-0.006857380270957947}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:14,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4640d7a44b2f6cb5eae699f90c14ecc9: Running coprocessor pre-open hook at 1675846454760Writing region info on filesystem at 1675846454760Initializing all the Stores at 1675846454761 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846454761Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846454762 (+1 ms)Cleaning up temporary data from old regions at 1675846454877 (+115 ms)Cleaning up detritus from prior splits at 1675846454878 (+1 ms)Running coprocessor post-open hooks at 1675846454881 (+3 ms)Region opened successfully at 1675846454881 2023-02-08 08:54:14,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9., pid=33, masterSystemTime=1675846454654 2023-02-08 08:54:14,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:14,883 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:14,883 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,883 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): 4640d7a44b2f6cb5eae699f90c14ecc9/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4640d7a44b2f6cb5eae699f90c14ecc9/cf1 in testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,883 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): 4640d7a44b2f6cb5eae699f90c14ecc9/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4640d7a44b2f6cb5eae699f90c14ecc9/cf2 in testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/.tmp, totalSize=34.9 K 2023-02-08 08:54:14,883 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/.tmp, totalSize=34.9 K 2023-02-08 08:54:14,884 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846449309 2023-02-08 08:54:14,884 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846449309 2023-02-08 08:54:14,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:14,885 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846449508 2023-02-08 08:54:14,885 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846449508 2023-02-08 08:54:14,885 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846449712 2023-02-08 08:54:14,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4640d7a44b2f6cb5eae699f90c14ecc9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:14,885 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846449712 2023-02-08 08:54:14,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846454885"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846454885"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846454885"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454885"}]},"ts":"1675846454885"} 2023-02-08 08:54:14,886 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846449910 2023-02-08 08:54:14,886 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846449910 2023-02-08 08:54:14,886 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846450517 2023-02-08 08:54:14,886 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846450517 2023-02-08 08:54:14,887 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846450723 2023-02-08 08:54:14,887 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846450723 2023-02-08 08:54:14,887 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846450920 2023-02-08 08:54:14,888 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846450920 2023-02-08 08:54:14,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-08 08:54:14,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 4640d7a44b2f6cb5eae699f90c14ecc9, server=jenkins-hbase4.apache.org,44651,1675846441425 in 385 msec 2023-02-08 08:54:14,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-08 08:54:14,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, ASSIGN in 546 msec 2023-02-08 08:54:14,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 in 3.7110 sec 2023-02-08 08:54:14,903 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): 4640d7a44b2f6cb5eae699f90c14ecc9#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:14,904 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4640d7a44b2f6cb5eae699f90c14ecc9#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:14,919 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/.tmp/cf1/b8c5c330c1de4aedba8aeb7f04b79d85 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/b8c5c330c1de4aedba8aeb7f04b79d85 2023-02-08 08:54:14,919 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/.tmp/cf2/2c6f802e8069444ab157bdf58705a544 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/2c6f802e8069444ab157bdf58705a544 2023-02-08 08:54:14,925 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4640d7a44b2f6cb5eae699f90c14ecc9/cf2 of 4640d7a44b2f6cb5eae699f90c14ecc9 into 2c6f802e8069444ab157bdf58705a544(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:14,925 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4640d7a44b2f6cb5eae699f90c14ecc9/cf1 of 4640d7a44b2f6cb5eae699f90c14ecc9 into b8c5c330c1de4aedba8aeb7f04b79d85(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:14,925 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4640d7a44b2f6cb5eae699f90c14ecc9: 2023-02-08 08:54:14,925 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4640d7a44b2f6cb5eae699f90c14ecc9: 2023-02-08 08:54:14,925 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9., storeName=4640d7a44b2f6cb5eae699f90c14ecc9/cf2, priority=9, startTime=1675846454883; duration=0sec 2023-02-08 08:54:14,925 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9., storeName=4640d7a44b2f6cb5eae699f90c14ecc9/cf1, priority=9, startTime=1675846454882; duration=0sec 2023-02-08 08:54:14,925 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,925 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:15,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44651] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:49624 deadline: 1675846515089, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf. is not online on jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:15,201 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:54:15,208 INFO [Listener at localhost/33105] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 08:54:15,210 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:54:15,226 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:15,226 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,227 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,228 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,228 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,229 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-08 08:54:15,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-08 08:54:15,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,232 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,232 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:15,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846455233"}]},"ts":"1675846455233"} 2023-02-08 08:54:15,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-08 08:54:15,237 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-08 08:54:15,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, UNASSIGN}] 2023-02-08 08:54:15,239 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, UNASSIGN 2023-02-08 08:54:15,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=df6b1a9175455ee27dbcef5fc4d3c6b6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:15,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846455239"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846455239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846455239"}]},"ts":"1675846455239"} 2023-02-08 08:54:15,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure df6b1a9175455ee27dbcef5fc4d3c6b6, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:15,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, UNASSIGN 2023-02-08 08:54:15,244 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4640d7a44b2f6cb5eae699f90c14ecc9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:15,244 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846455244"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846455244"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846455244"}]},"ts":"1675846455244"} 2023-02-08 08:54:15,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 4640d7a44b2f6cb5eae699f90c14ecc9, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:15,333 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,334 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:15,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:15,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4640d7a44b2f6cb5eae699f90c14ecc9, disabling compactions & flushes 2023-02-08 08:54:15,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:15,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:15,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. after waiting 0 ms 2023-02-08 08:54:15,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:15,404 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-bottom] to archive 2023-02-08 08:54:15,405 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:15,407 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,408 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,409 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,411 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,412 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,413 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,414 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,416 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-bottom] to archive 2023-02-08 08:54:15,416 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:15,418 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,419 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,420 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,421 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,422 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,423 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,424 DEBUG [StoreCloser-testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:54:15,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:15,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9. 2023-02-08 08:54:15,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4640d7a44b2f6cb5eae699f90c14ecc9: Waiting for close lock at 1675846455393Running coprocessor pre-close hooks at 1675846455393Disabling compacts and flushes for region at 1675846455393Disabling writes for close at 1675846455393Writing region close event to WAL at 1675846455425 (+32 ms)Running coprocessor post-close hooks at 1675846455428 (+3 ms)Closed at 1675846455428 2023-02-08 08:54:15,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:15,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:15,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing df6b1a9175455ee27dbcef5fc4d3c6b6, disabling compactions & flushes 2023-02-08 08:54:15,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:15,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:15,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. after waiting 0 ms 2023-02-08 08:54:15,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:15,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4640d7a44b2f6cb5eae699f90c14ecc9, regionState=CLOSED 2023-02-08 08:54:15,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846455430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846455430"}]},"ts":"1675846455430"} 2023-02-08 08:54:15,432 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a-top] to archive 2023-02-08 08:54:15,432 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:15,434 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-08 08:54:15,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 4640d7a44b2f6cb5eae699f90c14ecc9, server=jenkins-hbase4.apache.org,44651,1675846441425 in 187 msec 2023-02-08 08:54:15,436 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4640d7a44b2f6cb5eae699f90c14ecc9, UNASSIGN in 197 msec 2023-02-08 08:54:15,437 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,439 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,440 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,441 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,442 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,444 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0-top] to archive 2023-02-08 08:54:15,445 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:15,446 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,447 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,448 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,449 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,450 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,452 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,453 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:54:15,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:15,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6. 2023-02-08 08:54:15,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for df6b1a9175455ee27dbcef5fc4d3c6b6: Waiting for close lock at 1675846455430Running coprocessor pre-close hooks at 1675846455430Disabling compacts and flushes for region at 1675846455430Disabling writes for close at 1675846455430Writing region close event to WAL at 1675846455454 (+24 ms)Running coprocessor post-close hooks at 1675846455457 (+3 ms)Closed at 1675846455457 2023-02-08 08:54:15,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:15,459 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=df6b1a9175455ee27dbcef5fc4d3c6b6, regionState=CLOSED 2023-02-08 08:54:15,459 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846455459"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846455459"}]},"ts":"1675846455459"} 2023-02-08 08:54:15,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-08 08:54:15,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure df6b1a9175455ee27dbcef5fc4d3c6b6, server=jenkins-hbase4.apache.org,44651,1675846441425 in 220 msec 2023-02-08 08:54:15,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-08 08:54:15,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=df6b1a9175455ee27dbcef5fc4d3c6b6, UNASSIGN in 225 msec 2023-02-08 08:54:15,466 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846455466"}]},"ts":"1675846455466"} 2023-02-08 08:54:15,467 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-08 08:54:15,469 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-08 08:54:15,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 241 msec 2023-02-08 08:54:15,535 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,535 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:15,536 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-08 08:54:15,536 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,537 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-08 08:54:15,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,540 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,540 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,540 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,541 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 08:54:15,547 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,547 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:15,547 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:15,549 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/recovered.edits] 2023-02-08 08:54:15,550 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/recovered.edits] 2023-02-08 08:54:15,550 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/recovered.edits] 2023-02-08 08:54:15,553 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/db33cd3302fe4b60a86e367cde8dd31c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/db33cd3302fe4b60a86e367cde8dd31c 2023-02-08 08:54:15,553 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/.regioninfo to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/.regioninfo 2023-02-08 08:54:15,554 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/19c94d08c0894d929430093894dc8665 2023-02-08 08:54:15,556 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/684935ab2c704e78ada3ff4b86eb9f65 2023-02-08 08:54:15,557 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/f941d7e3f54045b18928406200e8be16 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/f941d7e3f54045b18928406200e8be16 2023-02-08 08:54:15,557 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,558 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/6fde51fd1aca4f8dab5aa05553a96d18 2023-02-08 08:54:15,559 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,560 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/9929c7ad984b4653877770308e0d27a4 2023-02-08 08:54:15,561 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/.regioninfo to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/.regioninfo 2023-02-08 08:54:15,561 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,561 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/d6d6229f1845417cb8191b47f1edaf81 2023-02-08 08:54:15,563 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/da4cd6f12ee34be0a61c986fb38dfc5a 2023-02-08 08:54:15,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,564 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/19c94d08c0894d929430093894dc8665.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,565 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf1/f1b73fb138ca4985810d3feea95d63c2 2023-02-08 08:54:15,566 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,566 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/684935ab2c704e78ada3ff4b86eb9f65.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,568 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/6fde51fd1aca4f8dab5aa05553a96d18.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,569 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/37128f1bad744f44be02e014491f2f96 2023-02-08 08:54:15,570 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/9929c7ad984b4653877770308e0d27a4.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,570 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/7233ec322e644286b4c67bc51b7f7cc7 2023-02-08 08:54:15,571 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/d6d6229f1845417cb8191b47f1edaf81.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,572 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,572 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/72a864cd3d1743c2967fc73e4cc9cce5 2023-02-08 08:54:15,573 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/da4cd6f12ee34be0a61c986fb38dfc5a.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,573 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,574 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/815c0408cd2e414aa8b2cbbb84133a26 2023-02-08 08:54:15,575 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf1/f1b73fb138ca4985810d3feea95d63c2.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,575 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,576 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/a6ea560f0e4049b0afe2e4a910503ba0 2023-02-08 08:54:15,577 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,578 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e4996ac184ea43f9ba5b52b4c96a650e 2023-02-08 08:54:15,579 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,579 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/37128f1bad744f44be02e014491f2f96.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,580 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/cf2/e5c3e74c15b24592a24ff53722f3fb2c 2023-02-08 08:54:15,581 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/7233ec322e644286b4c67bc51b7f7cc7.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,581 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,583 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/72a864cd3d1743c2967fc73e4cc9cce5.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,583 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,584 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf/recovered.edits/85.seqid 2023-02-08 08:54:15,585 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,585 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/815c0408cd2e414aa8b2cbbb84133a26.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,587 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/a6ea560f0e4049b0afe2e4a910503ba0.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,587 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/b8c5c330c1de4aedba8aeb7f04b79d85 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf1/b8c5c330c1de4aedba8aeb7f04b79d85 2023-02-08 08:54:15,588 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e4996ac184ea43f9ba5b52b4c96a650e.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,589 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/2c6f802e8069444ab157bdf58705a544 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/cf2/2c6f802e8069444ab157bdf58705a544 2023-02-08 08:54:15,590 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/df6b1a9175455ee27dbcef5fc4d3c6b6/cf2/e5c3e74c15b24592a24ff53722f3fb2c.4c3aeb6b729e6681efb5e73977d7b2cf 2023-02-08 08:54:15,593 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9/recovered.edits/90.seqid 2023-02-08 08:54:15,594 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6/recovered.edits/90.seqid 2023-02-08 08:54:15,594 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/4640d7a44b2f6cb5eae699f90c14ecc9 2023-02-08 08:54:15,594 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitWithoutPONR/df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:15,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 08:54:15,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,600 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-08 08:54:15,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-08 08:54:15,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,604 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-08 08:54:15,604 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846455604"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:15,604 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846455604"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:15,604 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846455604"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:15,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:54:15,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4c3aeb6b729e6681efb5e73977d7b2cf, NAME => 'testSplitWithoutPONR,,1675846448797.4c3aeb6b729e6681efb5e73977d7b2cf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4640d7a44b2f6cb5eae699f90c14ecc9, NAME => 'testSplitWithoutPONR,,1675846451183.4640d7a44b2f6cb5eae699f90c14ecc9.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => df6b1a9175455ee27dbcef5fc4d3c6b6, NAME => 'testSplitWithoutPONR,41,1675846451183.df6b1a9175455ee27dbcef5fc4d3c6b6.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 08:54:15,606 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-08 08:54:15,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846455606"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:15,607 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-08 08:54:15,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:15,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 72 msec 2023-02-08 08:54:15,641 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,642 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 08:54:15,643 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-08 08:54:15,660 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=405 (was 393) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:49264 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45894 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45662 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:44651-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45858 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:37594 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:37534 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:49528 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:56620 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:49170 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45340 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:37544 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45910 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45674 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:43786 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45402 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:49512 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=870 (was 865) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=321 (was 288) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4338 (was 4488) 2023-02-08 08:54:15,674 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=405, OpenFileDescriptor=870, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=168, AvailableMemoryMB=4338 2023-02-08 08:54:15,674 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:15,675 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,675 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:15,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:15,679 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:15,681 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:15,682 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:15,684 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:15,684 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 empty. 2023-02-08 08:54:15,685 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:15,685 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 08:54:15,695 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:15,697 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 14b4c549942f07985130891b41879838, NAME => 'testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:15,705 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:15,705 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 14b4c549942f07985130891b41879838, disabling compactions & flushes 2023-02-08 08:54:15,706 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:15,706 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:15,706 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. after waiting 0 ms 2023-02-08 08:54:15,706 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:15,706 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:15,706 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 14b4c549942f07985130891b41879838: Waiting for close lock at 1675846455705Disabling compacts and flushes for region at 1675846455705Disabling writes for close at 1675846455706 (+1 ms)Writing region close event to WAL at 1675846455706Closed at 1675846455706 2023-02-08 08:54:15,708 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:15,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846455709"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846455709"}]},"ts":"1675846455709"} 2023-02-08 08:54:15,711 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:15,711 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:15,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846455711"}]},"ts":"1675846455711"} 2023-02-08 08:54:15,713 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 08:54:15,716 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:15,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:15,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:15,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:15,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:15,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN}] 2023-02-08 08:54:15,719 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:15,720 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:15,870 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:15,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:15,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846455871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846455871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846455871"}]},"ts":"1675846455871"} 2023-02-08 08:54:15,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:16,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 14b4c549942f07985130891b41879838, NAME => 'testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:16,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,030 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,032 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1 2023-02-08 08:54:16,032 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1 2023-02-08 08:54:16,032 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14b4c549942f07985130891b41879838 columnFamilyName cf1 2023-02-08 08:54:16,033 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(310): Store=14b4c549942f07985130891b41879838/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:16,033 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,034 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2 2023-02-08 08:54:16,034 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2 2023-02-08 08:54:16,035 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14b4c549942f07985130891b41879838 columnFamilyName cf2 2023-02-08 08:54:16,035 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(310): Store=14b4c549942f07985130891b41879838/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:16,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:16,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:16,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:16,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 14b4c549942f07985130891b41879838 2023-02-08 08:54:16,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:16,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 14b4c549942f07985130891b41879838; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60398340, jitterRate=-0.09999459981918335}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:16,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 14b4c549942f07985130891b41879838: Running coprocessor pre-open hook at 1675846456029Writing region info on filesystem at 1675846456029Initializing all the Stores at 1675846456030 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846456030Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846456030Cleaning up temporary data from old regions at 1675846456038 (+8 ms)Cleaning up detritus from prior splits at 1675846456038Running coprocessor post-open hooks at 1675846456049 (+11 ms)Region opened successfully at 1675846456049 2023-02-08 08:54:16,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838., pid=43, masterSystemTime=1675846456025 2023-02-08 08:54:16,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846456052"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846456052"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846456052"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846456052"}]},"ts":"1675846456052"} 2023-02-08 08:54:16,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-08 08:54:16,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 in 180 msec 2023-02-08 08:54:16,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-08 08:54:16,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN in 337 msec 2023-02-08 08:54:16,058 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:16,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846456058"}]},"ts":"1675846456058"} 2023-02-08 08:54:16,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 08:54:16,065 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:16,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 387 msec 2023-02-08 08:54:16,188 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,188 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:16,190 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,190 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,190 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:16,190 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:16,191 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,191 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,193 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,193 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,193 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:16,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,196 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,196 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:16,197 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:54:16,197 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,197 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,197 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,208 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/86026ed5f5894f5aa472dd734013ab99 2023-02-08 08:54:16,221 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/47fa16b72e3246488b126b073f3fc801 2023-02-08 08:54:16,226 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/86026ed5f5894f5aa472dd734013ab99 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 2023-02-08 08:54:16,230 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:16,231 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/47fa16b72e3246488b126b073f3fc801 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 2023-02-08 08:54:16,234 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:16,235 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 38ms, sequenceid=6, compaction requested=false 2023-02-08 08:54:16,235 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,235 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:16,236 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,236 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:16,236 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:16,236 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,236 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,236 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,237 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,237 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,238 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,238 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:16,239 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:16,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f7ff3d7[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:16,240 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,243 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:16,242 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:16,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:16,243 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:16,244 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,244 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,244 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,245 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,245 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:16,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:16,249 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:16,249 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:16,249 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:16,249 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:16,249 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:16,250 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,250 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,251 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,250 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,251 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,251 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,254 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,255 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,256 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,257 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,257 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:16,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,262 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,261 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,261 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,262 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,262 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:16,262 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,263 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,363 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,363 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,363 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,380 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,380 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:16,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:16,382 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,382 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:16,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:16,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,384 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,385 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,385 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,386 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:16,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,387 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,387 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,387 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,389 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,390 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,390 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,390 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,390 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,390 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,398 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/f49ab407cc004c04ba78b879e2fa900c 2023-02-08 08:54:16,413 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/0c71c015664d40e3ad35b609d58463f5 2023-02-08 08:54:16,418 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/f49ab407cc004c04ba78b879e2fa900c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c 2023-02-08 08:54:16,422 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:16,423 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/0c71c015664d40e3ad35b609d58463f5 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 2023-02-08 08:54:16,428 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:16,429 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 42ms, sequenceid=19, compaction requested=false 2023-02-08 08:54:16,429 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:16,429 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,431 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,431 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,432 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,432 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:16,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78e047ec[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:16,434 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:16,434 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,435 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,436 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,436 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,435 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:16,436 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:16,437 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,437 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,437 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,441 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,441 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:16,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:16,445 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:16,446 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:16,446 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:16,446 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:16,446 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,448 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,449 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,450 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,451 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,453 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,453 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,455 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:16,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:16,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:16,459 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,460 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,560 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,560 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,560 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,575 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,575 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:16,577 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,577 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,578 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:16,578 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:16,578 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,578 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,579 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,580 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,580 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,588 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:16,588 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,588 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,588 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:16,588 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:16,589 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,600 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/8abdc73284aa41a28ab07ac349f29042 2023-02-08 08:54:16,614 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/bb4ca12920ef44ef9a2eb3379d59d91e 2023-02-08 08:54:16,619 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/8abdc73284aa41a28ab07ac349f29042 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 2023-02-08 08:54:16,623 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:16,624 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/bb4ca12920ef44ef9a2eb3379d59d91e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e 2023-02-08 08:54:16,629 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:16,630 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 42ms, sequenceid=32, compaction requested=true 2023-02-08 08:54:16,630 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:16,630 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,632 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:16,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57b11265[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:16,634 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:16,634 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,636 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,636 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:16,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,636 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,637 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,638 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,638 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,639 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,639 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,639 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,639 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:16,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:16,642 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:16,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:16,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:16,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:16,642 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,644 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,645 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,645 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,647 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,648 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,648 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,655 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,656 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:16,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,655 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,656 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:16,656 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,656 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,656 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,756 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,756 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,756 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,772 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,772 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:16,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:16,774 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,774 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,774 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:16,774 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:16,775 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,775 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,777 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,777 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,777 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,779 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,780 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:16,780 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:16,780 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,781 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,781 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,781 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,781 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,781 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,789 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/74d55fb691764a519b35243cdd4a07d6 2023-02-08 08:54:16,801 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/b67643d9fb7a415f99e9d4526dabb061 2023-02-08 08:54:16,807 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/74d55fb691764a519b35243cdd4a07d6 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 2023-02-08 08:54:16,811 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:16,812 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/b67643d9fb7a415f99e9d4526dabb061 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 2023-02-08 08:54:16,817 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:16,818 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 38ms, sequenceid=45, compaction requested=true 2023-02-08 08:54:16,818 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:16,818 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,819 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,819 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,820 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,820 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:16,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fc8b9c1[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:16,822 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:16,822 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,824 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:16,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:16,825 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:16,826 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,826 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,826 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:16,827 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,827 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:16,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:16,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:16,830 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:16,830 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:16,830 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:16,830 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:16,830 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,832 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,832 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,833 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,833 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,835 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:16,835 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,835 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,838 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,839 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,840 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,940 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,940 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,940 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,955 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,955 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:16,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:16,957 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:16,957 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,957 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:16,957 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:16,958 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,958 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,959 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,959 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:16,960 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,962 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:16,962 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:16,962 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:16,962 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:16,963 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,965 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,965 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,965 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,965 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:16,965 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,972 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/f59bb9b2544f466dad37c336e7b218d6 2023-02-08 08:54:16,983 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/2b40d6de378a48f7948d207c93c05a0c 2023-02-08 08:54:16,988 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/f59bb9b2544f466dad37c336e7b218d6 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 2023-02-08 08:54:16,993 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:16,994 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/2b40d6de378a48f7948d207c93c05a0c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c 2023-02-08 08:54:16,999 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:17,000 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 38ms, sequenceid=58, compaction requested=true 2023-02-08 08:54:17,000 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:17,000 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:17,000 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:17,000 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:17,000 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,000 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:17,000 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:17,003 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:17,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f5900a4[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:17,006 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:17,006 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,007 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,007 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:17,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,008 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:17,008 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,008 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:17,010 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,010 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,013 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:17,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:17,016 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:17,016 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:17,016 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:17,016 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:17,016 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,018 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,019 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,019 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,021 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,022 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,022 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,027 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:17,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:17,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,028 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:17,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,027 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,028 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,029 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,129 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:17,129 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,129 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:17,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:17,145 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,145 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:17,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:17,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:17,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 08:54:17,148 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 08:54:17,148 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:17,148 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 08:54:17,148 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:54:17,148 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,148 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,150 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,150 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 08:54:17,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:17,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:17,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 08:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:17,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:17,152 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:17,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:17,153 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. started... 2023-02-08 08:54:17,153 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:17,154 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,155 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:17,162 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/ba57b526a0124f2aa69685b052a9e67b 2023-02-08 08:54:17,175 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/c21272c9afaf4238998ec9d3265075d5 2023-02-08 08:54:17,179 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/ba57b526a0124f2aa69685b052a9e67b as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b 2023-02-08 08:54:17,184 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:17,185 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/c21272c9afaf4238998ec9d3265075d5 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 2023-02-08 08:54:17,189 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:17,190 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 37ms, sequenceid=71, compaction requested=true 2023-02-08 08:54:17,190 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:17,190 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 08:54:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 08:54:17,193 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 08:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@157ce584[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:17,195 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:54:17,195 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,196 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,196 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:17,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,196 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,197 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,197 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:54:17,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,197 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 08:54:17,198 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:17,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,199 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,200 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,202 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 08:54:17,202 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,202 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 08:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:54:17,205 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 08:54:17,205 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:17,205 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:17,205 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 08:54:17,205 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,208 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,208 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:17,209 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:17,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 08:54:17,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 08:54:17,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:17,217 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:17,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,218 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:17,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,218 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:17,217 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:17,218 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,218 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:17,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:17,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:17,218 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 08:54:17,219 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 08:54:17,319 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:17,319 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,319 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:17,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:17,581 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 08:54:17,582 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:17,583 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,584 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,584 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,584 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,585 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:17,585 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:17,585 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:17,833 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,833 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:17,833 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:17,834 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:17,834 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-08 08:54:17,834 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:17,834 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:17,834 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:17,834 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:17,835 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:17,836 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 08:54:17,836 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:17,837 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:17,837 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:17,837 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:17,838 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:17,838 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:17,839 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:17,839 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:17,839 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:17,839 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:17,839 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:17,840 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:17,840 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:17,840 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,840 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:17,840 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,840 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,841 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,841 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:17,841 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:17,842 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:17,842 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846487842 2023-02-08 08:54:17,842 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:17,844 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,846 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:17,846 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:17,846 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:17,846 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846517846 2023-02-08 08:54:17,846 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846577846 2023-02-08 08:54:17,846 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:17,847 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:17,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:17,849 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:17,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:17,849 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:18,097 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,097 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:18,097 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:18,097 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:18,098 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:18,098 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:18,098 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:18,098 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:18,098 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:18,099 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:18,100 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 08:54:18,100 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:18,101 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:18,101 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:18,101 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:18,101 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,102 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:18,102 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,102 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,102 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,102 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:18,103 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,103 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,103 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:18,103 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,103 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,103 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,103 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,103 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,104 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:18,104 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:18,105 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:18,105 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846488105 2023-02-08 08:54:18,106 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:18,107 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,109 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:18,109 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:18,110 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:18,110 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846518110 2023-02-08 08:54:18,110 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846578110 2023-02-08 08:54:18,110 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:18,110 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:18,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN}] 2023-02-08 08:54:18,113 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:18,113 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:18,113 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:18,360 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,360 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:18,360 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:18,360 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:18,363 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:18,363 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:18,363 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:18,363 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:18,363 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:18,364 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:18,365 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 08:54:18,365 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:18,365 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:18,365 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:18,365 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:18,366 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,366 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:18,366 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,366 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,367 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,367 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:18,367 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,368 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,368 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:18,368 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,368 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,369 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,369 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,369 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,369 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:18,369 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:18,370 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:18,370 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846488370 2023-02-08 08:54:18,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,371 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:18,373 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:18,373 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:18,373 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:18,373 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846518373 2023-02-08 08:54:18,374 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846578373 2023-02-08 08:54:18,374 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:18,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN}] 2023-02-08 08:54:18,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:18,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846458376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846458376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846458376"}]},"ts":"1675846458376"} 2023-02-08 08:54:18,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:18,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:18,378 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:18,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:18,624 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,624 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:18,624 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:18,624 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:18,625 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:18,626 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:18,626 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:18,626 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:18,626 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:18,627 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:18,628 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:18,628 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:18,628 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:18,629 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:18,629 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:18,629 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:18,630 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,630 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:18,631 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,631 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,631 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,632 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:18,632 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,632 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,633 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:18,634 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,634 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,634 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,634 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,634 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,634 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,634 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,634 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,635 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 08:54:18,635 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:18,636 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:18,636 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:18,636 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846488636 2023-02-08 08:54:18,636 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:18,637 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,639 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:18,639 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:18,639 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:18,639 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846518639 2023-02-08 08:54:18,640 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846578640 2023-02-08 08:54:18,640 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:18,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846458640"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846458640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846458640"}]},"ts":"1675846458640"} 2023-02-08 08:54:18,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:18,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:18,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 14b4c549942f07985130891b41879838 2023-02-08 08:54:18,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 14b4c549942f07985130891b41879838, disabling compactions & flushes 2023-02-08 08:54:18,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:18,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:18,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. after waiting 0 ms 2023-02-08 08:54:18,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:18,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 14b4c549942f07985130891b41879838 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:54:18,805 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/19ea80df56f04e779ae855ad36679c1f 2023-02-08 08:54:18,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/8deea30a629d42698efeab49ea0902ec 2023-02-08 08:54:18,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/19ea80df56f04e779ae855ad36679c1f as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f 2023-02-08 08:54:18,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/8deea30a629d42698efeab49ea0902ec as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec 2023-02-08 08:54:18,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:54:18,831 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 36ms, sequenceid=82, compaction requested=true 2023-02-08 08:54:18,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:54:18,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:18,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:18,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 14b4c549942f07985130891b41879838: Waiting for close lock at 1675846458795Running coprocessor pre-close hooks at 1675846458795Disabling compacts and flushes for region at 1675846458795Disabling writes for close at 1675846458795Obtaining lock to block concurrent updates at 1675846458795Preparing flush snapshotting stores in 14b4c549942f07985130891b41879838 at 1675846458795Finished memstore snapshotting testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846458795Flushing stores of testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. at 1675846458796 (+1 ms)Flushing 14b4c549942f07985130891b41879838/cf1: creating writer at 1675846458797 (+1 ms)Flushing 14b4c549942f07985130891b41879838/cf1: appending metadata at 1675846458799 (+2 ms)Flushing 14b4c549942f07985130891b41879838/cf1: closing flushed file at 1675846458799Flushing 14b4c549942f07985130891b41879838/cf2: creating writer at 1675846458810 (+11 ms)Flushing 14b4c549942f07985130891b41879838/cf2: appending metadata at 1675846458811 (+1 ms)Flushing 14b4c549942f07985130891b41879838/cf2: closing flushed file at 1675846458811Flushing 14b4c549942f07985130891b41879838/cf1: reopening flushed file at 1675846458822 (+11 ms)Flushing 14b4c549942f07985130891b41879838/cf2: reopening flushed file at 1675846458827 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 14b4c549942f07985130891b41879838 in 36ms, sequenceid=82, compaction requested=true at 1675846458831 (+4 ms)Writing region close event to WAL at 1675846458833 (+2 ms)Running coprocessor post-close hooks at 1675846458836 (+3 ms)Closed at 1675846458836 2023-02-08 08:54:18,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 14b4c549942f07985130891b41879838 2023-02-08 08:54:18,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSED 2023-02-08 08:54:18,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846458839"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846458839"}]},"ts":"1675846458839"} 2023-02-08 08:54:18,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:18,841 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:18,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:18,890 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,890 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:18,890 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:18,890 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:18,891 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:18,891 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:18,891 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:18,891 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:18,891 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:18,892 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:18,893 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:18,894 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:18,894 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,894 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:18,894 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,894 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:18,895 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,895 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:18,895 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,895 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:18,895 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:18,896 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,896 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,896 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,896 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,896 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,896 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,896 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:18,896 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,897 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,897 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:18,897 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:18,897 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:18,898 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:18,898 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:18,898 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846488898 2023-02-08 08:54:18,898 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:18,899 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,901 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:18,901 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:18,901 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:18,901 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846518901 2023-02-08 08:54:18,901 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846578901 2023-02-08 08:54:18,902 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:18,902 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:18,902 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846458902"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846458902"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846458902"}]},"ts":"1675846458902"} 2023-02-08 08:54:18,903 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:18,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-08 08:54:18,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 in 261 msec 2023-02-08 08:54:18,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:18,906 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:18,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:18,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:19,152 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,152 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:19,152 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:19,152 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:19,153 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:19,153 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:19,153 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:19,153 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:19,154 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:19,154 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:19,155 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:19,155 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 08:54:19,155 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:19,156 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:19,156 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:19,156 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:19,157 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:19,157 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,157 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:19,157 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,157 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,158 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,158 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:19,158 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,158 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,158 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:19,159 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,159 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:19,159 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:19,160 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:19,160 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,160 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,160 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,160 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:19,160 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:19,160 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:19,161 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:19,161 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:19,161 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:19,161 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846489161 2023-02-08 08:54:19,161 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:19,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:19,165 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:19,165 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:19,165 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:19,165 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846519165 2023-02-08 08:54:19,165 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846579165 2023-02-08 08:54:19,165 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:19,165 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:19,167 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:19,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-08 08:54:19,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN in 793 msec 2023-02-08 08:54:19,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:19,172 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:19,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:19,415 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,415 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:19,416 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:19,416 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:19,417 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:19,417 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:19,417 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:19,417 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:19,417 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:19,418 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:19,418 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:19,419 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:19,420 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,420 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:19,420 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,420 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:19,421 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,421 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:19,421 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,421 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:19,421 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:19,422 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:19,422 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:19,423 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:19,424 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:19,424 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846489424 2023-02-08 08:54:19,425 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,425 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:19,428 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:19,428 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:19,428 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:19,428 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846519428 2023-02-08 08:54:19,428 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846579428 2023-02-08 08:54:19,428 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:19,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:19,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:19,443 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=14b4c549942f07985130891b41879838, threads=14 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,444 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,448 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,448 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,448 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,448 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,452 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,452 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,452 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,453 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,579 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,586 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,593 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,596 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,596 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,603 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,605 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,605 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,605 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,614 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,624 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,625 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:19,628 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,026 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,026 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 14b4c549942f07985130891b41879838 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:20,076 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:20,076 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:20,076 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:20,160 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:20,178 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:20,179 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:20,179 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:20,180 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:20,181 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 08:54:20,181 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:20,181 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:20,181 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:20,182 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:20,182 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:20,182 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:20,182 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:20,183 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:20,184 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:20,185 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,185 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,185 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,185 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,185 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,185 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:20,185 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:20,186 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:20,187 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846490187 2023-02-08 08:54:20,189 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:20,189 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,192 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:20,192 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:20,192 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:20,192 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846520192 2023-02-08 08:54:20,192 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846580192 2023-02-08 08:54:20,193 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:54:20,204 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=14b4c549942f07985130891b41879838, threads=14 2023-02-08 08:54:20,206 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,206 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,210 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,210 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,210 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,212 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,217 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,217 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,217 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,218 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,218 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,220 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,220 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,224 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,351 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,352 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,354 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,354 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,357 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,364 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,392 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,400 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,404 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,404 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,407 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,412 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,412 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,414 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 for region: 14b4c549942f07985130891b41879838 2023-02-08 08:54:20,415 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 14b4c549942f07985130891b41879838 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:20,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:20,461 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/7c2d1fee6eaa17c701bd1dfe9d595207/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:20,462 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/b319ee264d2fa548e3cb0e2cf2a76764/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:54:20,463 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:20,463 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:20,463 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:20,693 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,693 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:20,693 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:20,693 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:20,694 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:20,694 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:20,694 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:20,694 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:20,694 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:20,695 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:20,696 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 08:54:20,696 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:20,696 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:20,697 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:20,697 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:20,698 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:20,698 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:20,698 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,699 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:20,699 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,699 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,700 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,700 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:20,700 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,700 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,701 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:20,702 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,702 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:20,702 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:20,704 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:20,704 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,704 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,704 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,705 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,705 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,705 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 08:54:20,706 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:20,706 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:20,706 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846490706 2023-02-08 08:54:20,708 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:20,708 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,710 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:20,710 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:20,711 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:20,711 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846520711 2023-02-08 08:54:20,711 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846580711 2023-02-08 08:54:20,711 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:20,713 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:20,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:20,716 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:20,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:20,961 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:20,962 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:20,962 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:20,963 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:20,964 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 08:54:20,964 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:20,964 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:20,964 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:20,965 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:20,965 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:20,965 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:20,965 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:20,966 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,967 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:20,967 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:20,968 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,968 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,968 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:20,968 DEBUG [Listener at localhost/33105] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,968 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 08:54:20,969 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:20,969 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:20,970 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846490970 2023-02-08 08:54:20,970 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:20,973 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:20,975 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:20,975 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:20,975 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:20,975 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846520975 2023-02-08 08:54:20,975 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846580975 2023-02-08 08:54:20,975 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:54:20,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:20,988 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:20,988 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:20,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:20,988 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:21,226 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,226 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:21,226 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:21,227 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:21,227 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:21,227 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:21,227 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:21,227 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:21,227 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:21,228 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:21,229 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 08:54:21,229 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:21,229 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:21,230 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:21,230 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:21,230 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:21,231 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:21,231 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,231 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:21,232 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,232 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,232 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,232 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:21,233 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,233 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,233 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:21,234 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,234 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:21,234 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:21,234 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:21,234 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,234 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,235 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,235 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,235 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,235 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:54:21,236 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:21,236 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:21,236 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846491236 2023-02-08 08:54:21,236 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:21,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,242 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:21,242 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:21,242 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:21,242 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846521242 2023-02-08 08:54:21,242 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846581242 2023-02-08 08:54:21,242 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:54:21,242 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:21,245 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:21,245 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:21,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:21,245 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:21,493 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,493 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:21,493 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:21,494 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:21,494 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:21,494 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:21,494 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:21,494 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:21,495 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:21,495 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:21,496 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 08:54:21,496 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:21,496 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:21,496 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:21,497 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:21,497 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:21,497 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:21,497 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,497 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,498 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,499 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:21,499 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,499 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:21,499 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:21,500 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:21,500 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,500 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,500 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,500 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,500 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,500 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:21,501 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:21,501 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:21,501 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846491501 2023-02-08 08:54:21,502 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:21,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,505 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:21,505 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:21,505 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:21,505 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846521505 2023-02-08 08:54:21,505 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846581505 2023-02-08 08:54:21,505 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:21,505 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:21,507 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:21,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:21,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:21,507 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:21,755 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,755 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:21,756 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:21,756 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:21,757 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:21,757 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:21,757 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:21,757 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:21,757 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:21,758 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:21,758 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 08:54:21,758 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:21,758 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:21,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:21,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:21,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:21,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:21,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:21,762 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:21,762 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,762 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,762 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,762 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,762 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:21,762 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 08:54:21,763 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:21,763 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:21,763 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846491763 2023-02-08 08:54:21,764 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:21,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:21,767 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:21,768 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:21,768 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:21,768 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846521768 2023-02-08 08:54:21,768 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846581768 2023-02-08 08:54:21,768 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:54:21,768 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:21,772 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN] 2023-02-08 08:54:21,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:21,772 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:21,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:21,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:22,018 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:22,019 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:22,019 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:22,020 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:22,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 08:54:22,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:54:22,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:54:22,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:22,021 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:22,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:22,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:22,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:22,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:22,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:22,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:22,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:22,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,027 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:54:22,027 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:22,028 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:22,028 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:22,029 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846492029 2023-02-08 08:54:22,029 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:22,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,032 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,034 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:22,034 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:22,034 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:22,034 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846522034 2023-02-08 08:54:22,034 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846582034 2023-02-08 08:54:22,034 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:22,034 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:22,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:22,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:22,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:22,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:22,284 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,284 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:22,284 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:22,284 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:22,287 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:22,287 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:22,287 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:22,287 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:22,287 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:22,288 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:22,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:22,290 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,290 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:22,290 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,290 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,290 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,291 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,292 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:22,292 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,292 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,292 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,292 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,292 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 08:54:22,292 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:22,293 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:22,293 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:22,293 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846492293 2023-02-08 08:54:22,294 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:22,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,296 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,298 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:22,298 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:22,298 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:22,298 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846522298 2023-02-08 08:54:22,298 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846582298 2023-02-08 08:54:22,298 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:22,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:22,300 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:22,300 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:22,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:22,300 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:22,548 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:22,549 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:22,549 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:22,550 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:22,550 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 08:54:22,551 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:22,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:22,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:22,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:22,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,553 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:22,553 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,553 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,554 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,554 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,554 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 08:54:22,554 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:22,555 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:22,555 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:22,555 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846492555 2023-02-08 08:54:22,556 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:22,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,558 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,560 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:22,560 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:22,560 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:22,560 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846522560 2023-02-08 08:54:22,560 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846582560 2023-02-08 08:54:22,560 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:54:22,560 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:22,562 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:22,562 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 08:54:22,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:22,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:22,810 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,810 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:22,811 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:22,811 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:22,812 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:22,812 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 08:54:22,812 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 08:54:22,813 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:22,813 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:22,813 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:22,813 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:22,813 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:22,814 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,815 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:22,815 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:22,815 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,815 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,815 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:22,815 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,815 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,815 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,816 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:54:22,816 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 08:54:22,816 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:22,817 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:22,817 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:22,817 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846492817 2023-02-08 08:54:22,819 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:22,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:22,821 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,823 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:22,824 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:22,824 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:22,824 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846522824 2023-02-08 08:54:22,824 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846582824 2023-02-08 08:54:22,824 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:54:22,824 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:54:22,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 exec-time=5.2420 sec 2023-02-08 08:54:22,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,827 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=true 2023-02-08 08:54:22,978 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:22,979 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:22,979 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846462979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846462979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846462979"}]},"ts":"1675846462979"} 2023-02-08 08:54:22,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:22,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:54:22,981 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:22,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:54:22,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:54:23,074 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:23,074 WARN [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-08 08:54:23,074 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:54:23,074 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:54:23,074 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:54:23,075 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:54:23,075 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:54:23,075 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:54:23,075 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:54:23,075 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:54:23,075 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:54:23,076 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-08 08:54:23,077 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:54:23,077 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:54:23,077 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:54:23,077 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:54:23,077 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:23,078 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:54:23,079 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:23,079 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:54:23,079 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:23,079 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:54:23,079 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:23,079 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 08:54:23,080 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:54:23,080 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:54:23,080 INFO [Listener at localhost/33105] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838 to restore RIT 2023-02-08 08:54:23,081 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846493081 2023-02-08 08:54:23,082 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:54:23,082 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:54:23,083 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:54:23,084 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=14b4c549942f07985130891b41879838, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,44651,1675846441425, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425, openSeqNum=2 2023-02-08 08:54:23,084 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:54:23,084 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846523084 2023-02-08 08:54:23,084 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846583084 2023-02-08 08:54:23,084 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 08:54:23,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:23,085 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846463084"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846463084"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846463084"}]},"ts":"1675846463084"} 2023-02-08 08:54:23,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:23,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 14b4c549942f07985130891b41879838, NAME => 'testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:23,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,243 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,244 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1 2023-02-08 08:54:23,244 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1 2023-02-08 08:54:23,244 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14b4c549942f07985130891b41879838 columnFamilyName cf1 2023-02-08 08:54:23,250 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f 2023-02-08 08:54:23,253 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 2023-02-08 08:54:23,256 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 2023-02-08 08:54:23,259 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 2023-02-08 08:54:23,262 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b 2023-02-08 08:54:23,265 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c 2023-02-08 08:54:23,268 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 2023-02-08 08:54:23,268 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(310): Store=14b4c549942f07985130891b41879838/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:23,268 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,269 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2 2023-02-08 08:54:23,269 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2 2023-02-08 08:54:23,270 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14b4c549942f07985130891b41879838 columnFamilyName cf2 2023-02-08 08:54:23,276 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 2023-02-08 08:54:23,280 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c 2023-02-08 08:54:23,283 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 2023-02-08 08:54:23,287 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec 2023-02-08 08:54:23,290 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 2023-02-08 08:54:23,293 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e 2023-02-08 08:54:23,296 DEBUG [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 2023-02-08 08:54:23,297 INFO [StoreOpener-14b4c549942f07985130891b41879838-1] regionserver.HStore(310): Store=14b4c549942f07985130891b41879838/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:23,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:23,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:23,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.splits 2023-02-08 08:54:23,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:23,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 14b4c549942f07985130891b41879838; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69435558, jitterRate=0.03467044234275818}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:23,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 14b4c549942f07985130891b41879838: Running coprocessor pre-open hook at 1675846463242Writing region info on filesystem at 1675846463242Initializing all the Stores at 1675846463243 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846463243Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846463243Cleaning up temporary data from old regions at 1675846463299 (+56 ms)Cleaning up detritus from prior splits at 1675846463299Running coprocessor post-open hooks at 1675846463303 (+4 ms)Region opened successfully at 1675846463303 2023-02-08 08:54:23,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838., pid=48, masterSystemTime=1675846463238 2023-02-08 08:54:23,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:23,305 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:23,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:23,305 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:23,306 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 08:54:23,306 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 08:54:23,306 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): 14b4c549942f07985130891b41879838/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:23,306 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): 14b4c549942f07985130891b41879838/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:23,306 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 14b4c549942f07985130891b41879838/cf2 in testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,306 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 14b4c549942f07985130891b41879838/cf1 in testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,306 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp, totalSize=34.9 K 2023-02-08 08:54:23,306 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp, totalSize=34.9 K 2023-02-08 08:54:23,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 47fa16b72e3246488b126b073f3fc801, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846456183 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 86026ed5f5894f5aa472dd734013ab99, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846456183 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 0c71c015664d40e3ad35b609d58463f5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846456365 2023-02-08 08:54:23,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting f49ab407cc004c04ba78b879e2fa900c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846456365 2023-02-08 08:54:23,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846463308"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846463308"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846463308"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846463308"}]},"ts":"1675846463308"} 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting bb4ca12920ef44ef9a2eb3379d59d91e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846456562 2023-02-08 08:54:23,308 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 8abdc73284aa41a28ab07ac349f29042, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846456562 2023-02-08 08:54:23,309 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting b67643d9fb7a415f99e9d4526dabb061, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846456758 2023-02-08 08:54:23,309 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 74d55fb691764a519b35243cdd4a07d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846456758 2023-02-08 08:54:23,309 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 2b40d6de378a48f7948d207c93c05a0c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846456942 2023-02-08 08:54:23,310 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting f59bb9b2544f466dad37c336e7b218d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846456942 2023-02-08 08:54:23,310 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting c21272c9afaf4238998ec9d3265075d5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846457131 2023-02-08 08:54:23,310 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting ba57b526a0124f2aa69685b052a9e67b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846457131 2023-02-08 08:54:23,310 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 8deea30a629d42698efeab49ea0902ec, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846457320 2023-02-08 08:54:23,310 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 19ea80df56f04e779ae855ad36679c1f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846457320 2023-02-08 08:54:23,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-08 08:54:23,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 in 224 msec 2023-02-08 08:54:23,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN in 1.5420 sec 2023-02-08 08:54:23,329 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): 14b4c549942f07985130891b41879838#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:23,332 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): 14b4c549942f07985130891b41879838#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:23,348 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf2/3d77902ebd9b4a4cb82657e9b30e474e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/3d77902ebd9b4a4cb82657e9b30e474e 2023-02-08 08:54:23,349 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/.tmp/cf1/b6526d09a1c44884b7e05778d86b298f as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/b6526d09a1c44884b7e05778d86b298f 2023-02-08 08:54:23,357 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 14b4c549942f07985130891b41879838/cf1 of 14b4c549942f07985130891b41879838 into b6526d09a1c44884b7e05778d86b298f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:23,357 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 14b4c549942f07985130891b41879838/cf2 of 14b4c549942f07985130891b41879838 into 3d77902ebd9b4a4cb82657e9b30e474e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:23,357 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:23,357 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 14b4c549942f07985130891b41879838: 2023-02-08 08:54:23,357 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838., storeName=14b4c549942f07985130891b41879838/cf1, priority=9, startTime=1675846463304; duration=0sec 2023-02-08 08:54:23,357 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838., storeName=14b4c549942f07985130891b41879838/cf2, priority=9, startTime=1675846463305; duration=0sec 2023-02-08 08:54:23,358 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:23,358 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:23,661 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-08 08:54:23,663 DEBUG [Listener at localhost/33105] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-08 08:54:23,665 INFO [Listener at localhost/33105] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 08:54:23,667 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-08 08:54:23,668 INFO [Listener at localhost/33105] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 08:54:23,668 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-08 08:54:23,685 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:23,685 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,685 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,686 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,686 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,687 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-08 08:54:23,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-08 08:54:23,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:23,690 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,690 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 08:54:23,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846463691"}]},"ts":"1675846463691"} 2023-02-08 08:54:23,692 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 08:54:23,694 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-08 08:54:23,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN}] 2023-02-08 08:54:23,695 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN 2023-02-08 08:54:23,695 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:23,695 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846463695"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846463695"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846463695"}]},"ts":"1675846463695"} 2023-02-08 08:54:23,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:23,791 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,791 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 08:54:23,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 14b4c549942f07985130891b41879838, disabling compactions & flushes 2023-02-08 08:54:23,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. after waiting 0 ms 2023-02-08 08:54:23,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,849 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f] to archive 2023-02-08 08:54:23,850 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:23,851 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/86026ed5f5894f5aa472dd734013ab99 2023-02-08 08:54:23,852 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f49ab407cc004c04ba78b879e2fa900c 2023-02-08 08:54:23,854 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/8abdc73284aa41a28ab07ac349f29042 2023-02-08 08:54:23,855 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/74d55fb691764a519b35243cdd4a07d6 2023-02-08 08:54:23,856 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/f59bb9b2544f466dad37c336e7b218d6 2023-02-08 08:54:23,857 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/ba57b526a0124f2aa69685b052a9e67b 2023-02-08 08:54:23,859 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/19ea80df56f04e779ae855ad36679c1f 2023-02-08 08:54:23,863 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec] to archive 2023-02-08 08:54:23,863 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:23,865 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/47fa16b72e3246488b126b073f3fc801 2023-02-08 08:54:23,866 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/0c71c015664d40e3ad35b609d58463f5 2023-02-08 08:54:23,867 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/bb4ca12920ef44ef9a2eb3379d59d91e 2023-02-08 08:54:23,868 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/b67643d9fb7a415f99e9d4526dabb061 2023-02-08 08:54:23,869 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/2b40d6de378a48f7948d207c93c05a0c 2023-02-08 08:54:23,870 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/c21272c9afaf4238998ec9d3265075d5 2023-02-08 08:54:23,871 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/8deea30a629d42698efeab49ea0902ec 2023-02-08 08:54:23,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:54:23,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:23,875 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838. 2023-02-08 08:54:23,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 14b4c549942f07985130891b41879838: Waiting for close lock at 1675846463848Running coprocessor pre-close hooks at 1675846463848Disabling compacts and flushes for region at 1675846463848Disabling writes for close at 1675846463848Writing region close event to WAL at 1675846463872 (+24 ms)Running coprocessor post-close hooks at 1675846463875 (+3 ms)Closed at 1675846463875 2023-02-08 08:54:23,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 14b4c549942f07985130891b41879838 2023-02-08 08:54:23,877 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=14b4c549942f07985130891b41879838, regionState=CLOSED 2023-02-08 08:54:23,877 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846463877"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846463877"}]},"ts":"1675846463877"} 2023-02-08 08:54:23,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-08 08:54:23,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 14b4c549942f07985130891b41879838, server=jenkins-hbase4.apache.org,44651,1675846441425 in 182 msec 2023-02-08 08:54:23,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-08 08:54:23,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, UNASSIGN in 186 msec 2023-02-08 08:54:23,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846463882"}]},"ts":"1675846463882"} 2023-02-08 08:54:23,883 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 08:54:23,884 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-08 08:54:23,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 198 msec 2023-02-08 08:54:23,992 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,992 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 08:54:23,993 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-08 08:54:23,994 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,994 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-08 08:54:23,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:23,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:23,997 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,997 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:23,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 08:54:24,001 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:24,003 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits] 2023-02-08 08:54:24,005 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/b6526d09a1c44884b7e05778d86b298f to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf1/b6526d09a1c44884b7e05778d86b298f 2023-02-08 08:54:24,008 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/3d77902ebd9b4a4cb82657e9b30e474e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/cf2/3d77902ebd9b4a4cb82657e9b30e474e 2023-02-08 08:54:24,010 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838/recovered.edits/90.seqid 2023-02-08 08:54:24,011 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackAndDoubleExecution/14b4c549942f07985130891b41879838 2023-02-08 08:54:24,011 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 08:54:24,014 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:24,016 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-08 08:54:24,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-08 08:54:24,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:24,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-08 08:54:24,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846464018"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:24,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 08:54:24,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 14b4c549942f07985130891b41879838, NAME => 'testRollbackAndDoubleExecution,,1675846455678.14b4c549942f07985130891b41879838.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 08:54:24,020 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-08 08:54:24,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846464020"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:24,021 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-08 08:54:24,023 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:54:24,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-02-08 08:54:24,098 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:24,098 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:24,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 08:54:24,099 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-08 08:54:24,113 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401 (was 405), OpenFileDescriptor=868 (was 870), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=287 (was 321), ProcessCount=168 (was 168), AvailableMemoryMB=4318 (was 4338) 2023-02-08 08:54:24,127 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401, OpenFileDescriptor=868, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=168, AvailableMemoryMB=4317 2023-02-08 08:54:24,127 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:24,127 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:24,127 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:24,129 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:24,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:24,131 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:24,133 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:24,133 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:24,135 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,135 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 empty. 2023-02-08 08:54:24,136 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,136 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 08:54:24,148 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:24,149 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 63d7e5dfe0dba860d1578c977cb72017, NAME => 'testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:24,163 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:24,163 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 63d7e5dfe0dba860d1578c977cb72017, disabling compactions & flushes 2023-02-08 08:54:24,163 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,163 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,163 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. after waiting 0 ms 2023-02-08 08:54:24,164 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,164 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,164 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 63d7e5dfe0dba860d1578c977cb72017: Waiting for close lock at 1675846464163Disabling compacts and flushes for region at 1675846464163Disabling writes for close at 1675846464164 (+1 ms)Writing region close event to WAL at 1675846464164Closed at 1675846464164 2023-02-08 08:54:24,166 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:24,166 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464166"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846464166"}]},"ts":"1675846464166"} 2023-02-08 08:54:24,168 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:24,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:24,169 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846464169"}]},"ts":"1675846464169"} 2023-02-08 08:54:24,170 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-08 08:54:24,174 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:24,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:24,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:24,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:24,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:24,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, ASSIGN}] 2023-02-08 08:54:24,176 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, ASSIGN 2023-02-08 08:54:24,177 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:24,327 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:24,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=63d7e5dfe0dba860d1578c977cb72017, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,328 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464328"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846464328"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846464328"}]},"ts":"1675846464328"} 2023-02-08 08:54:24,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 63d7e5dfe0dba860d1578c977cb72017, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:24,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 63d7e5dfe0dba860d1578c977cb72017, NAME => 'testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:24,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:24,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:24,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,490 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,491 DEBUG [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf1 2023-02-08 08:54:24,491 DEBUG [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf1 2023-02-08 08:54:24,492 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63d7e5dfe0dba860d1578c977cb72017 columnFamilyName cf1 2023-02-08 08:54:24,492 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] regionserver.HStore(310): Store=63d7e5dfe0dba860d1578c977cb72017/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:24,492 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,494 DEBUG [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf2 2023-02-08 08:54:24,494 DEBUG [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf2 2023-02-08 08:54:24,494 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63d7e5dfe0dba860d1578c977cb72017 columnFamilyName cf2 2023-02-08 08:54:24,495 INFO [StoreOpener-63d7e5dfe0dba860d1578c977cb72017-1] regionserver.HStore(310): Store=63d7e5dfe0dba860d1578c977cb72017/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:24,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:24,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:24,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 63d7e5dfe0dba860d1578c977cb72017; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71996300, jitterRate=0.07282847166061401}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:24,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 63d7e5dfe0dba860d1578c977cb72017: Running coprocessor pre-open hook at 1675846464488Writing region info on filesystem at 1675846464488Initializing all the Stores at 1675846464489 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846464489Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846464489Cleaning up temporary data from old regions at 1675846464497 (+8 ms)Cleaning up detritus from prior splits at 1675846464498 (+1 ms)Running coprocessor post-open hooks at 1675846464502 (+4 ms)Region opened successfully at 1675846464502 2023-02-08 08:54:24,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017., pid=55, masterSystemTime=1675846464481 2023-02-08 08:54:24,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,505 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=63d7e5dfe0dba860d1578c977cb72017, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,505 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464505"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846464505"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846464505"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846464505"}]},"ts":"1675846464505"} 2023-02-08 08:54:24,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-08 08:54:24,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 63d7e5dfe0dba860d1578c977cb72017, server=jenkins-hbase4.apache.org,39055,1675846441504 in 177 msec 2023-02-08 08:54:24,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-08 08:54:24,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, ASSIGN in 334 msec 2023-02-08 08:54:24,511 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:24,511 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846464511"}]},"ts":"1675846464511"} 2023-02-08 08:54:24,512 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-08 08:54:24,515 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:24,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 385 msec 2023-02-08 08:54:24,632 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,633 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:24,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:24,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:24,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:24,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, UNASSIGN}] 2023-02-08 08:54:24,637 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, UNASSIGN 2023-02-08 08:54:24,637 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=63d7e5dfe0dba860d1578c977cb72017, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,637 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464637"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846464637"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846464637"}]},"ts":"1675846464637"} 2023-02-08 08:54:24,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 63d7e5dfe0dba860d1578c977cb72017, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:24,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 63d7e5dfe0dba860d1578c977cb72017, disabling compactions & flushes 2023-02-08 08:54:24,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. after waiting 0 ms 2023-02-08 08:54:24,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 08:54:24,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:24,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017. 2023-02-08 08:54:24,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 63d7e5dfe0dba860d1578c977cb72017: Waiting for close lock at 1675846464791Running coprocessor pre-close hooks at 1675846464791Disabling compacts and flushes for region at 1675846464791Disabling writes for close at 1675846464792 (+1 ms)Writing region close event to WAL at 1675846464792Running coprocessor post-close hooks at 1675846464795 (+3 ms)Closed at 1675846464795 2023-02-08 08:54:24,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:24,797 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=63d7e5dfe0dba860d1578c977cb72017, regionState=CLOSED 2023-02-08 08:54:24,797 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464797"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846464797"}]},"ts":"1675846464797"} 2023-02-08 08:54:24,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-08 08:54:24,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 63d7e5dfe0dba860d1578c977cb72017, server=jenkins-hbase4.apache.org,39055,1675846441504 in 160 msec 2023-02-08 08:54:24,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-08 08:54:24,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=63d7e5dfe0dba860d1578c977cb72017, UNASSIGN in 164 msec 2023-02-08 08:54:24,826 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 08:54:24,828 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 08:54:24,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846464829"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675846464829"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675846464829"}]},"ts":"1675846464829"} 2023-02-08 08:54:24,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846464829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846464829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846464829"}]},"ts":"1675846464829"} 2023-02-08 08:54:24,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846464829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846464829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846464829"}]},"ts":"1675846464829"} 2023-02-08 08:54:24,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, ASSIGN}] 2023-02-08 08:54:24,833 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, ASSIGN 2023-02-08 08:54:24,834 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:24,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, ASSIGN 2023-02-08 08:54:24,835 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:24,984 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:54:24,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=01f3ff9798c228c40b5ec93eee2dc700, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846464986"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846464986"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846464986"}]},"ts":"1675846464986"} 2023-02-08 08:54:24,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 01f3ff9798c228c40b5ec93eee2dc700, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:24,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=81a836dbf579d2f8cf4ba6c294cf81ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:24,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846464990"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846464990"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846464990"}]},"ts":"1675846464990"} 2023-02-08 08:54:24,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 81a836dbf579d2f8cf4ba6c294cf81ed, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:25,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 01f3ff9798c228c40b5ec93eee2dc700, NAME => 'testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 08:54:25,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:25,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:25,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,144 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,145 DEBUG [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf1 2023-02-08 08:54:25,145 DEBUG [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf1 2023-02-08 08:54:25,146 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01f3ff9798c228c40b5ec93eee2dc700 columnFamilyName cf1 2023-02-08 08:54:25,146 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] regionserver.HStore(310): Store=01f3ff9798c228c40b5ec93eee2dc700/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:25,146 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,147 DEBUG [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf2 2023-02-08 08:54:25,147 DEBUG [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf2 2023-02-08 08:54:25,148 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01f3ff9798c228c40b5ec93eee2dc700 columnFamilyName cf2 2023-02-08 08:54:25,148 INFO [StoreOpener-01f3ff9798c228c40b5ec93eee2dc700-1] regionserver.HStore(310): Store=01f3ff9798c228c40b5ec93eee2dc700/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:25,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:25,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 01f3ff9798c228c40b5ec93eee2dc700; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74465783, jitterRate=0.1096266359090805}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:25,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 01f3ff9798c228c40b5ec93eee2dc700: Running coprocessor pre-open hook at 1675846465143Writing region info on filesystem at 1675846465143Initializing all the Stores at 1675846465144 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846465144Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846465144Cleaning up temporary data from old regions at 1675846465151 (+7 ms)Cleaning up detritus from prior splits at 1675846465151Running coprocessor post-open hooks at 1675846465153 (+2 ms)Region opened successfully at 1675846465153 2023-02-08 08:54:25,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700., pid=61, masterSystemTime=1675846465139 2023-02-08 08:54:25,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 81a836dbf579d2f8cf4ba6c294cf81ed, NAME => 'testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 08:54:25,156 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=01f3ff9798c228c40b5ec93eee2dc700, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:25,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:25,156 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465156"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846465156"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846465156"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846465156"}]},"ts":"1675846465156"} 2023-02-08 08:54:25,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:25,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,157 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,159 DEBUG [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf1 2023-02-08 08:54:25,159 DEBUG [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf1 2023-02-08 08:54:25,159 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 81a836dbf579d2f8cf4ba6c294cf81ed columnFamilyName cf1 2023-02-08 08:54:25,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-08 08:54:25,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 01f3ff9798c228c40b5ec93eee2dc700, server=jenkins-hbase4.apache.org,39055,1675846441504 in 170 msec 2023-02-08 08:54:25,160 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] regionserver.HStore(310): Store=81a836dbf579d2f8cf4ba6c294cf81ed/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:25,160 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, ASSIGN in 328 msec 2023-02-08 08:54:25,161 DEBUG [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf2 2023-02-08 08:54:25,161 DEBUG [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf2 2023-02-08 08:54:25,161 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 81a836dbf579d2f8cf4ba6c294cf81ed columnFamilyName cf2 2023-02-08 08:54:25,162 INFO [StoreOpener-81a836dbf579d2f8cf4ba6c294cf81ed-1] regionserver.HStore(310): Store=81a836dbf579d2f8cf4ba6c294cf81ed/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:25,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:25,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 81a836dbf579d2f8cf4ba6c294cf81ed; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70796605, jitterRate=0.05495162308216095}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:25,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 81a836dbf579d2f8cf4ba6c294cf81ed: Running coprocessor pre-open hook at 1675846465156Writing region info on filesystem at 1675846465156Initializing all the Stores at 1675846465157 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846465157Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846465157Cleaning up temporary data from old regions at 1675846465164 (+7 ms)Cleaning up detritus from prior splits at 1675846465165 (+1 ms)Running coprocessor post-open hooks at 1675846465168 (+3 ms)Region opened successfully at 1675846465168 2023-02-08 08:54:25,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed., pid=62, masterSystemTime=1675846465139 2023-02-08 08:54:25,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,171 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=81a836dbf579d2f8cf4ba6c294cf81ed, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:25,171 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465171"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846465171"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846465171"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846465171"}]},"ts":"1675846465171"} 2023-02-08 08:54:25,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-08 08:54:25,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 81a836dbf579d2f8cf4ba6c294cf81ed, server=jenkins-hbase4.apache.org,39055,1675846441504 in 182 msec 2023-02-08 08:54:25,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-08 08:54:25,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, ASSIGN in 343 msec 2023-02-08 08:54:25,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 in 544 msec 2023-02-08 08:54:25,387 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:54:25,390 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:25,390 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,390 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,391 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,391 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,392 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,395 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,395 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 08:54:25,396 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846465396"}]},"ts":"1675846465396"} 2023-02-08 08:54:25,397 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-08 08:54:25,398 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-08 08:54:25,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, UNASSIGN}] 2023-02-08 08:54:25,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, UNASSIGN 2023-02-08 08:54:25,400 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=01f3ff9798c228c40b5ec93eee2dc700, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:25,400 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465400"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846465400"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846465400"}]},"ts":"1675846465400"} 2023-02-08 08:54:25,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 01f3ff9798c228c40b5ec93eee2dc700, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:25,403 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, UNASSIGN 2023-02-08 08:54:25,404 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=81a836dbf579d2f8cf4ba6c294cf81ed, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:25,404 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465404"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846465404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846465404"}]},"ts":"1675846465404"} 2023-02-08 08:54:25,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 81a836dbf579d2f8cf4ba6c294cf81ed, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:25,496 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,496 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 08:54:25,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 01f3ff9798c228c40b5ec93eee2dc700, disabling compactions & flushes 2023-02-08 08:54:25,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. after waiting 0 ms 2023-02-08 08:54:25,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 08:54:25,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:25,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700. 2023-02-08 08:54:25,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 01f3ff9798c228c40b5ec93eee2dc700: Waiting for close lock at 1675846465553Running coprocessor pre-close hooks at 1675846465553Disabling compacts and flushes for region at 1675846465553Disabling writes for close at 1675846465554 (+1 ms)Writing region close event to WAL at 1675846465554Running coprocessor post-close hooks at 1675846465559 (+5 ms)Closed at 1675846465559 2023-02-08 08:54:25,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 81a836dbf579d2f8cf4ba6c294cf81ed, disabling compactions & flushes 2023-02-08 08:54:25,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. after waiting 0 ms 2023-02-08 08:54:25,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=01f3ff9798c228c40b5ec93eee2dc700, regionState=CLOSED 2023-02-08 08:54:25,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846465562"}]},"ts":"1675846465562"} 2023-02-08 08:54:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-08 08:54:25,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 01f3ff9798c228c40b5ec93eee2dc700, server=jenkins-hbase4.apache.org,39055,1675846441504 in 162 msec 2023-02-08 08:54:25,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 08:54:25,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:25,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed. 2023-02-08 08:54:25,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 81a836dbf579d2f8cf4ba6c294cf81ed: Waiting for close lock at 1675846465561Running coprocessor pre-close hooks at 1675846465561Disabling compacts and flushes for region at 1675846465561Disabling writes for close at 1675846465562 (+1 ms)Writing region close event to WAL at 1675846465562Running coprocessor post-close hooks at 1675846465566 (+4 ms)Closed at 1675846465566 2023-02-08 08:54:25,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=01f3ff9798c228c40b5ec93eee2dc700, UNASSIGN in 166 msec 2023-02-08 08:54:25,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,567 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=81a836dbf579d2f8cf4ba6c294cf81ed, regionState=CLOSED 2023-02-08 08:54:25,567 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846465567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846465567"}]},"ts":"1675846465567"} 2023-02-08 08:54:25,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-08 08:54:25,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 81a836dbf579d2f8cf4ba6c294cf81ed, server=jenkins-hbase4.apache.org,39055,1675846441504 in 163 msec 2023-02-08 08:54:25,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-08 08:54:25,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=81a836dbf579d2f8cf4ba6c294cf81ed, UNASSIGN in 171 msec 2023-02-08 08:54:25,572 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846465572"}]},"ts":"1675846465572"} 2023-02-08 08:54:25,573 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-08 08:54:25,575 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-08 08:54:25,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 183 msec 2023-02-08 08:54:25,697 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,697 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 08:54:25,698 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-08 08:54:25,698 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,698 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,701 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,701 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,701 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,702 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 08:54:25,707 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:25,707 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,707 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,709 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/recovered.edits] 2023-02-08 08:54:25,709 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/recovered.edits] 2023-02-08 08:54:25,710 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/recovered.edits] 2023-02-08 08:54:25,718 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/recovered.edits/7.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed/recovered.edits/7.seqid 2023-02-08 08:54:25,719 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/recovered.edits/4.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017/recovered.edits/4.seqid 2023-02-08 08:54:25,719 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/81a836dbf579d2f8cf4ba6c294cf81ed 2023-02-08 08:54:25,720 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/63d7e5dfe0dba860d1578c977cb72017 2023-02-08 08:54:25,720 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/recovered.edits/7.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700/recovered.edits/7.seqid 2023-02-08 08:54:25,720 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionNoStoreFile/01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:54:25,720 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 08:54:25,723 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,725 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-08 08:54:25,727 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-08 08:54:25,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-08 08:54:25,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846465728"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:25,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846465728"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:25,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846465728"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:25,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:54:25,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 63d7e5dfe0dba860d1578c977cb72017, NAME => 'testSplitTableRegionNoStoreFile,,1675846464130.63d7e5dfe0dba860d1578c977cb72017.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 81a836dbf579d2f8cf4ba6c294cf81ed, NAME => 'testSplitTableRegionNoStoreFile,,1675846464632.81a836dbf579d2f8cf4ba6c294cf81ed.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 01f3ff9798c228c40b5ec93eee2dc700, NAME => 'testSplitTableRegionNoStoreFile,41,1675846464632.01f3ff9798c228c40b5ec93eee2dc700.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 08:54:25,730 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-08 08:54:25,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846465730"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:25,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-08 08:54:25,734 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:54:25,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-02-08 08:54:25,802 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,803 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 08:54:25,804 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-08 08:54:25,818 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 401) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_713569061_15 at /127.0.0.1:45662 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_713569061_15 at /127.0.0.1:45340 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=870 (was 868) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=287 (was 287), ProcessCount=168 (was 168), AvailableMemoryMB=4315 (was 4317) 2023-02-08 08:54:25,833 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=870, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=168, AvailableMemoryMB=4312 2023-02-08 08:54:25,833 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:25,833 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,833 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,835 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:25,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:25,837 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:25,839 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:25,840 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:25,842 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:25,842 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d empty. 2023-02-08 08:54:25,843 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:25,843 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 08:54:25,864 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:25,865 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2c6d5653f20710b77eab401d7e0f610d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 2c6d5653f20710b77eab401d7e0f610d, disabling compactions & flushes 2023-02-08 08:54:25,897 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. after waiting 0 ms 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:25,897 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:25,897 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 2c6d5653f20710b77eab401d7e0f610d: Waiting for close lock at 1675846465897Disabling compacts and flushes for region at 1675846465897Disabling writes for close at 1675846465897Writing region close event to WAL at 1675846465897Closed at 1675846465897 2023-02-08 08:54:25,902 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:25,903 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846465903"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846465903"}]},"ts":"1675846465903"} 2023-02-08 08:54:25,905 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:25,906 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:25,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846465906"}]},"ts":"1675846465906"} 2023-02-08 08:54:25,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-08 08:54:25,910 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:25,912 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:25,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, ASSIGN}] 2023-02-08 08:54:25,915 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, ASSIGN 2023-02-08 08:54:25,916 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:26,066 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:26,068 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=2c6d5653f20710b77eab401d7e0f610d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846466068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846466068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846466068"}]},"ts":"1675846466068"} 2023-02-08 08:54:26,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 2c6d5653f20710b77eab401d7e0f610d, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:26,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2c6d5653f20710b77eab401d7e0f610d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:26,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,226 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,228 DEBUG [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1 2023-02-08 08:54:26,228 DEBUG [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1 2023-02-08 08:54:26,228 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c6d5653f20710b77eab401d7e0f610d columnFamilyName cf1 2023-02-08 08:54:26,229 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] regionserver.HStore(310): Store=2c6d5653f20710b77eab401d7e0f610d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:26,229 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,230 DEBUG [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2 2023-02-08 08:54:26,230 DEBUG [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2 2023-02-08 08:54:26,231 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c6d5653f20710b77eab401d7e0f610d columnFamilyName cf2 2023-02-08 08:54:26,231 INFO [StoreOpener-2c6d5653f20710b77eab401d7e0f610d-1] regionserver.HStore(310): Store=2c6d5653f20710b77eab401d7e0f610d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:26,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:26,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:26,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:26,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2c6d5653f20710b77eab401d7e0f610d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74022442, jitterRate=0.10302034020423889}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:26,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2c6d5653f20710b77eab401d7e0f610d: Running coprocessor pre-open hook at 1675846466225Writing region info on filesystem at 1675846466225Initializing all the Stores at 1675846466226 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846466226Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846466226Cleaning up temporary data from old regions at 1675846466233 (+7 ms)Cleaning up detritus from prior splits at 1675846466234 (+1 ms)Running coprocessor post-open hooks at 1675846466238 (+4 ms)Region opened successfully at 1675846466238 2023-02-08 08:54:26,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d., pid=71, masterSystemTime=1675846466221 2023-02-08 08:54:26,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=2c6d5653f20710b77eab401d7e0f610d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846466241"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846466241"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846466241"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846466241"}]},"ts":"1675846466241"} 2023-02-08 08:54:26,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-08 08:54:26,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 2c6d5653f20710b77eab401d7e0f610d, server=jenkins-hbase4.apache.org,44651,1675846441425 in 174 msec 2023-02-08 08:54:26,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-08 08:54:26,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, ASSIGN in 332 msec 2023-02-08 08:54:26,247 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:26,247 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846466247"}]},"ts":"1675846466247"} 2023-02-08 08:54:26,249 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-08 08:54:26,251 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:26,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 414 msec 2023-02-08 08:54:26,345 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,345 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:26,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,347 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,347 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,348 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:26,348 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:26,348 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,348 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,349 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,350 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,350 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,352 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,352 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:26,352 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:26,352 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,352 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:26,353 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:54:26,355 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,355 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,356 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,356 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,356 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,356 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,366 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/8a3e0a59b59a400daa52deca88c5a256 2023-02-08 08:54:26,381 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/68e2928405ee446db75123f9173e900c 2023-02-08 08:54:26,385 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/8a3e0a59b59a400daa52deca88c5a256 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256 2023-02-08 08:54:26,389 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:26,389 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/68e2928405ee446db75123f9173e900c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c 2023-02-08 08:54:26,393 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:54:26,394 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 41ms, sequenceid=6, compaction requested=false 2023-02-08 08:54:26,394 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,395 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:26,395 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,395 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,397 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,397 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:26,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a01ff74[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:26,399 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:26,399 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,400 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:26,401 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:26,403 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,403 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,403 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,404 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,405 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:26,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:26,407 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:26,407 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:26,407 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:26,407 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,407 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,410 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,410 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,410 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,412 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,412 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,413 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,420 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,420 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:26,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,421 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,422 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,422 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,422 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,521 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:26,522 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,522 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:26,535 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,535 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:26,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:26,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,538 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:26,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:26,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,538 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,540 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,540 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,540 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,540 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,540 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:26,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,541 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:26,542 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,542 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:26,542 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:26,543 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,551 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6886165c1b6d432c83e5eebaa918c599 2023-02-08 08:54:26,567 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/cc685eecccaa491893084f1d6b18b030 2023-02-08 08:54:26,571 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6886165c1b6d432c83e5eebaa918c599 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599 2023-02-08 08:54:26,575 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:26,576 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/cc685eecccaa491893084f1d6b18b030 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030 2023-02-08 08:54:26,580 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:54:26,581 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 39ms, sequenceid=19, compaction requested=false 2023-02-08 08:54:26,581 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:26,581 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,581 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:26,581 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:26,581 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,581 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,581 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,584 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,584 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:26,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10dd4144[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:26,586 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:26,586 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,587 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,588 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:26,587 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,587 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,588 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,589 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,590 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,590 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,590 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,590 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:26,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:26,594 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:26,594 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:26,594 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:26,594 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,594 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,599 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,599 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,600 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,601 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,602 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,603 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,606 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,605 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,606 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:26,607 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,707 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:26,707 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,707 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:26,720 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,720 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:26,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:26,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,722 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,723 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,723 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:26,723 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:26,723 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,723 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,724 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,724 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,724 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,724 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,725 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,725 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:26,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,727 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:26,727 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,727 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:26,728 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,729 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,741 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6b02bdc649164e80806fefaf53db6cb0 2023-02-08 08:54:26,753 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/f2678c6ee5184285886a0ae01b79ba52 2023-02-08 08:54:26,757 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6b02bdc649164e80806fefaf53db6cb0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0 2023-02-08 08:54:26,761 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:26,761 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/f2678c6ee5184285886a0ae01b79ba52 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52 2023-02-08 08:54:26,765 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:54:26,766 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 38ms, sequenceid=32, compaction requested=true 2023-02-08 08:54:26,766 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:26,766 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,766 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,771 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,771 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37a62fdb[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:26,773 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:26,773 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,774 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,774 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,775 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,775 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:26,775 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,776 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,777 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,777 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:26,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:26,780 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:26,780 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:26,780 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:26,780 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,780 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:26,782 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,782 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,782 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,782 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,782 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,783 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,783 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,783 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,784 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,785 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,785 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,786 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,788 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,789 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:26,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,788 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,789 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,790 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,889 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:26,890 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,890 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:26,903 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,904 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:26,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:26,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,906 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,906 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,906 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:26,906 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:26,906 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,906 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,908 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,908 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,908 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,908 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,908 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:26,910 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,910 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,910 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:26,910 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,919 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/cb7e43086314429da4fb9bd9d56c8eb5 2023-02-08 08:54:26,935 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6482253615544c0e8828d0b6eef93caf 2023-02-08 08:54:26,939 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/cb7e43086314429da4fb9bd9d56c8eb5 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5 2023-02-08 08:54:26,943 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:26,944 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6482253615544c0e8828d0b6eef93caf as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf 2023-02-08 08:54:26,948 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:54:26,949 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 39ms, sequenceid=45, compaction requested=true 2023-02-08 08:54:26,949 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:26,949 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:26,949 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:26,949 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:26,949 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,949 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:26,949 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:26,951 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,951 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:26,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4504ac6e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:26,953 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:26,953 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,954 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,954 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:26,954 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,955 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:26,955 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:26,957 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,957 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,958 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,958 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:26,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:26,961 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:26,962 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:26,962 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:26,962 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,962 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,966 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,967 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,968 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,969 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,969 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,970 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:26,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:26,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,973 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:26,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:26,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:26,974 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,074 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,074 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,075 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,087 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,087 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,089 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,089 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,089 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:27,089 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:27,090 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,090 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,091 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,092 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,092 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,094 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,093 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,094 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,094 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,094 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,094 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,094 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:27,094 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:27,095 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,106 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/05654d915cb74a7e92c06368a1417f4b 2023-02-08 08:54:27,117 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/b8db6e2a357d454fb8d27f95eeeba145 2023-02-08 08:54:27,122 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/05654d915cb74a7e92c06368a1417f4b as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b 2023-02-08 08:54:27,126 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:27,126 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/b8db6e2a357d454fb8d27f95eeeba145 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145 2023-02-08 08:54:27,130 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:54:27,131 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 37ms, sequenceid=58, compaction requested=true 2023-02-08 08:54:27,131 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:27,131 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,133 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,133 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,134 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,134 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bb01e5[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,136 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:27,136 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,137 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,137 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:27,138 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,138 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,138 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:27,139 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,139 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,139 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,141 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,145 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:27,145 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,145 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,145 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,145 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,147 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,147 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,152 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,154 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,154 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,154 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,157 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,158 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:27,157 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,158 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,258 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,259 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,259 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,270 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,270 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,273 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:27,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:27,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,278 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,278 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,279 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,279 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,279 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,280 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,280 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,281 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,282 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,282 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,282 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,282 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,282 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,288 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/d96339374f384183924736db96e99229 2023-02-08 08:54:27,307 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/e163002730b04997bbe6792da2c5bb23 2023-02-08 08:54:27,311 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/d96339374f384183924736db96e99229 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229 2023-02-08 08:54:27,315 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:27,315 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/e163002730b04997bbe6792da2c5bb23 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23 2023-02-08 08:54:27,319 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:54:27,320 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 41ms, sequenceid=71, compaction requested=true 2023-02-08 08:54:27,320 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:27,320 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,320 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,320 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,320 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,321 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,321 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,327 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,327 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,327 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,327 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:27,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a16b7d7[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:27,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,338 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:27,342 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,342 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,342 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,342 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,344 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:27,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,347 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:27,347 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,347 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,347 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,347 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,348 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,348 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,348 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,349 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,350 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,350 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,351 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,352 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,352 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,358 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,358 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,359 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,358 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,358 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,359 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,360 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,360 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,360 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,360 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,360 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,460 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,460 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,460 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,472 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,472 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,474 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,474 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,474 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:27,474 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:27,475 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,475 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,476 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,476 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,476 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,477 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,478 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,478 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,478 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,478 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,478 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:27,479 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-08 08:54:27,479 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,480 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,480 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,480 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,489 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:27,493 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:27,500 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:27,503 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:27,504 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/6311a7895a454195b70cc0f11769af79 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:27,507 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:27,507 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 08:54:27,508 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6c4124b2a7a94eb88733c56a614b7af3 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:27,511 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:27,511 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 08:54:27,512 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 33ms, sequenceid=83, compaction requested=true 2023-02-08 08:54:27,512 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:27,512 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,512 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,512 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,512 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,512 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,512 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,515 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,515 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,515 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,515 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:27,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c4897b9[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,517 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:27,517 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,518 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,519 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:27,520 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,520 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,520 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,525 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:27,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,528 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:27,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,528 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,530 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,531 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,531 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,533 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,533 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,533 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,536 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,535 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,536 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,537 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,637 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,637 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,637 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,645 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,645 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,647 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,647 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,647 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:27,647 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:27,648 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,648 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,649 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,649 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,649 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,651 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,651 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,651 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:27,651 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 08:54:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,660 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:27,664 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:27,671 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:27,674 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:27,675 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/d7d763547fd9464798dbed45d880d7a1 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:27,678 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:27,678 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 08:54:27,679 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/6364e3599ba14b148be20082f26c8f2a as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:27,682 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:27,683 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 08:54:27,683 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 32ms, sequenceid=91, compaction requested=true 2023-02-08 08:54:27,683 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:27,683 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,683 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,684 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,684 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,684 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,684 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,685 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,685 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c4f3382[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:27,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,688 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,688 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,688 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,689 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:27,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,689 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,691 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,692 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,692 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:27,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,695 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:27,695 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,695 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,695 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,695 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,697 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,696 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,697 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,699 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,699 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,699 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,706 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,705 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,706 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,707 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,806 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,807 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,807 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,814 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,814 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,817 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 08:54:27,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 08:54:27,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,817 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,818 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,818 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,819 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,819 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,820 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 08:54:27,821 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,821 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,821 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,821 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. started... 2023-02-08 08:54:27,821 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,830 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/58e6e7b00071413982667481a556e71e 2023-02-08 08:54:27,834 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e6e7b00071413982667481a556e71e 2023-02-08 08:54:27,842 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:27,846 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:27,847 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/58e6e7b00071413982667481a556e71e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e 2023-02-08 08:54:27,851 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e6e7b00071413982667481a556e71e 2023-02-08 08:54:27,851 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 08:54:27,852 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/a369aae42c7546b9aec167ceafa048e4 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:27,857 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:27,857 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 08:54:27,858 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 37ms, sequenceid=99, compaction requested=true 2023-02-08 08:54:27,858 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2c6d5653f20710b77eab401d7e0f610d: 2023-02-08 08:54:27,858 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:27,858 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,858 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,858 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,858 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 08:54:27,858 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 08:54:27,860 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 08:54:27,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4423bdc7[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,862 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 08:54:27,862 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:54:27,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,863 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 08:54:27,863 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,863 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:54:27,865 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,865 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,865 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,866 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,867 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 08:54:27,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 08:54:27,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:54:27,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,870 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 08:54:27,870 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,870 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,870 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,870 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,871 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,872 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,872 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,872 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,874 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,874 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,874 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,877 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 08:54:27,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:54:27,877 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,878 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,878 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,878 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:27,878 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,878 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:27,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,978 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,978 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,978 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,981 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,982 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:27,983 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:27,983 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:27,983 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:27,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, UNASSIGN}] 2023-02-08 08:54:27,985 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, UNASSIGN 2023-02-08 08:54:27,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=2c6d5653f20710b77eab401d7e0f610d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:27,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846467986"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846467986"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846467986"}]},"ts":"1675846467986"} 2023-02-08 08:54:27,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 2c6d5653f20710b77eab401d7e0f610d, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:28,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2c6d5653f20710b77eab401d7e0f610d, disabling compactions & flushes 2023-02-08 08:54:28,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:28,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:28,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. after waiting 0 ms 2023-02-08 08:54:28,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:28,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2c6d5653f20710b77eab401d7e0f610d 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-08 08:54:28,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf1/c31435c57f5443a79cfedc4f2b1b8f0c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 08:54:28,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.tmp/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 08:54:28,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 40ms, sequenceid=103, compaction requested=true 2023-02-08 08:54:28,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-08 08:54:28,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:28,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. 2023-02-08 08:54:28,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2c6d5653f20710b77eab401d7e0f610d: Waiting for close lock at 1675846468139Running coprocessor pre-close hooks at 1675846468139Disabling compacts and flushes for region at 1675846468139Disabling writes for close at 1675846468139Obtaining lock to block concurrent updates at 1675846468139Preparing flush snapshotting stores in 2c6d5653f20710b77eab401d7e0f610d at 1675846468139Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675846468140 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d. at 1675846468140Flushing 2c6d5653f20710b77eab401d7e0f610d/cf1: creating writer at 1675846468141 (+1 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf1: appending metadata at 1675846468143 (+2 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf1: closing flushed file at 1675846468143Flushing 2c6d5653f20710b77eab401d7e0f610d/cf2: creating writer at 1675846468154 (+11 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf2: appending metadata at 1675846468155 (+1 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf2: closing flushed file at 1675846468156 (+1 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf1: reopening flushed file at 1675846468171 (+15 ms)Flushing 2c6d5653f20710b77eab401d7e0f610d/cf2: reopening flushed file at 1675846468175 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 2c6d5653f20710b77eab401d7e0f610d in 40ms, sequenceid=103, compaction requested=true at 1675846468179 (+4 ms)Writing region close event to WAL at 1675846468181 (+2 ms)Running coprocessor post-close hooks at 1675846468185 (+4 ms)Closed at 1675846468185 2023-02-08 08:54:28,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=2c6d5653f20710b77eab401d7e0f610d, regionState=CLOSED 2023-02-08 08:54:28,187 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846468187"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846468187"}]},"ts":"1675846468187"} 2023-02-08 08:54:28,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-08 08:54:28,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 2c6d5653f20710b77eab401d7e0f610d, server=jenkins-hbase4.apache.org,44651,1675846441425 in 201 msec 2023-02-08 08:54:28,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-08 08:54:28,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c6d5653f20710b77eab401d7e0f610d, UNASSIGN in 204 msec 2023-02-08 08:54:28,205 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=2c6d5653f20710b77eab401d7e0f610d, threads=16 2023-02-08 08:54:28,206 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,206 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,206 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,206 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,206 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,210 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,210 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,210 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,211 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,211 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,212 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,212 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,244 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:28,245 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,246 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,246 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,246 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,250 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,256 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145, top=false 2023-02-08 08:54:28,256 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b, top=false 2023-02-08 08:54:28,259 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:28,261 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf, top=false 2023-02-08 08:54:28,261 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,261 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da, top=true 2023-02-08 08:54:28,261 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23, top=false 2023-02-08 08:54:28,262 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e6e7b00071413982667481a556e71e 2023-02-08 08:54:28,265 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:28,266 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e6e7b00071413982667481a556e71e 2023-02-08 08:54:28,267 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e, top=true 2023-02-08 08:54:28,277 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:28,279 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:28,279 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a, top=true 2023-02-08 08:54:28,281 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:28,281 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4, top=true 2023-02-08 08:54:28,316 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145 for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,316 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,316 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,316 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4 for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,324 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,325 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23 for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,325 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,325 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,325 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,325 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,326 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,332 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,335 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5, top=false 2023-02-08 08:54:28,336 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,357 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229, top=false 2023-02-08 08:54:28,358 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,359 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,359 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,359 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:28,366 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,366 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c, top=true 2023-02-08 08:54:28,368 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:28,368 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1, top=true 2023-02-08 08:54:28,373 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5 for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,379 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:28,379 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,392 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1 for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,392 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,392 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,394 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229 for child: b5a2b4f4020c2cbcbab2bc91399ebe33, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,397 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/.splits/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c for child: d8a3a28ddb9422c08833bf51afe408bf, parent: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,397 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,397 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,420 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,420 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,422 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:28,422 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,423 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,424 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,426 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,431 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3 for region: 2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,431 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 2c6d5653f20710b77eab401d7e0f610d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:54:28,475 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 08:54:28,477 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 08:54:28,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675846468479"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675846468479"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675846468479"}]},"ts":"1675846468479"} 2023-02-08 08:54:28,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468479"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846468479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846468479"}]},"ts":"1675846468479"} 2023-02-08 08:54:28,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468479"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846468479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846468479"}]},"ts":"1675846468479"} 2023-02-08 08:54:28,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, ASSIGN}] 2023-02-08 08:54:28,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, ASSIGN 2023-02-08 08:54:28,484 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:28,485 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, ASSIGN 2023-02-08 08:54:28,485 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:54:28,634 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:54:28,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b5a2b4f4020c2cbcbab2bc91399ebe33, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:28,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846468636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846468636"}]},"ts":"1675846468636"} 2023-02-08 08:54:28,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure b5a2b4f4020c2cbcbab2bc91399ebe33, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:28,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=d8a3a28ddb9422c08833bf51afe408bf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:28,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468640"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846468640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846468640"}]},"ts":"1675846468640"} 2023-02-08 08:54:28,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure d8a3a28ddb9422c08833bf51afe408bf, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:28,746 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:28,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b5a2b4f4020c2cbcbab2bc91399ebe33, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.', STARTKEY => '', ENDKEY => '60'} 2023-02-08 08:54:28,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:28,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:28,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,794 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,795 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 2023-02-08 08:54:28,795 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 2023-02-08 08:54:28,796 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5a2b4f4020c2cbcbab2bc91399ebe33 columnFamilyName cf1 2023-02-08 08:54:28,806 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,807 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-bottom 2023-02-08 08:54:28,811 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-bottom 2023-02-08 08:54:28,816 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-bottom 2023-02-08 08:54:28,820 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-bottom 2023-02-08 08:54:28,823 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b 2023-02-08 08:54:28,825 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5 2023-02-08 08:54:28,828 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229 2023-02-08 08:54:28,828 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(310): Store=b5a2b4f4020c2cbcbab2bc91399ebe33/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:28,828 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,829 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 2023-02-08 08:54:28,830 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 2023-02-08 08:54:28,830 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5a2b4f4020c2cbcbab2bc91399ebe33 columnFamilyName cf2 2023-02-08 08:54:28,839 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-bottom 2023-02-08 08:54:28,842 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,843 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-bottom 2023-02-08 08:54:28,846 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-bottom 2023-02-08 08:54:28,851 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-bottom 2023-02-08 08:54:28,854 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf 2023-02-08 08:54:28,857 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145 2023-02-08 08:54:28,860 DEBUG [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23 2023-02-08 08:54:28,860 INFO [StoreOpener-b5a2b4f4020c2cbcbab2bc91399ebe33-1] regionserver.HStore(310): Store=b5a2b4f4020c2cbcbab2bc91399ebe33/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:28,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:28,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:28,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b5a2b4f4020c2cbcbab2bc91399ebe33; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69914062, jitterRate=0.04180070757865906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:28,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b5a2b4f4020c2cbcbab2bc91399ebe33: Running coprocessor pre-open hook at 1675846468793Writing region info on filesystem at 1675846468793Initializing all the Stores at 1675846468794 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846468794Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846468794Cleaning up temporary data from old regions at 1675846468862 (+68 ms)Cleaning up detritus from prior splits at 1675846468862Running coprocessor post-open hooks at 1675846468865 (+3 ms)Region opened successfully at 1675846468865 2023-02-08 08:54:28,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33., pid=77, masterSystemTime=1675846468789 2023-02-08 08:54:28,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,867 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:28,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,867 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,867 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:28,867 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp, totalSize=35.0 K 2023-02-08 08:54:28,867 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp, totalSize=35.0 K 2023-02-08 08:54:28,868 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846466340 2023-02-08 08:54:28,868 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846466340 2023-02-08 08:54:28,868 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846466523 2023-02-08 08:54:28,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:28,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,869 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846466523 2023-02-08 08:54:28,869 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b5a2b4f4020c2cbcbab2bc91399ebe33, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:28,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d8a3a28ddb9422c08833bf51afe408bf, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.', STARTKEY => '60', ENDKEY => ''} 2023-02-08 08:54:28,869 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468869"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846468869"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846468869"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846468869"}]},"ts":"1675846468869"} 2023-02-08 08:54:28,869 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846466708 2023-02-08 08:54:28,869 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846466708 2023-02-08 08:54:28,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:28,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:28,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,870 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846466891 2023-02-08 08:54:28,870 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846467076 2023-02-08 08:54:28,871 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846466891 2023-02-08 08:54:28,871 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846467260 2023-02-08 08:54:28,871 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846467076 2023-02-08 08:54:28,872 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846467461 2023-02-08 08:54:28,872 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,872 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846467260 2023-02-08 08:54:28,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-08 08:54:28,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure b5a2b4f4020c2cbcbab2bc91399ebe33, server=jenkins-hbase4.apache.org,44651,1675846441425 in 233 msec 2023-02-08 08:54:28,873 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846467461 2023-02-08 08:54:28,875 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1 2023-02-08 08:54:28,875 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1 2023-02-08 08:54:28,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, ASSIGN in 391 msec 2023-02-08 08:54:28,875 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8a3a28ddb9422c08833bf51afe408bf columnFamilyName cf1 2023-02-08 08:54:28,886 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,887 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-top 2023-02-08 08:54:28,892 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-top 2023-02-08 08:54:28,894 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): b5a2b4f4020c2cbcbab2bc91399ebe33#cf2#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:28,895 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): b5a2b4f4020c2cbcbab2bc91399ebe33#cf1#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:28,901 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-top 2023-02-08 08:54:28,905 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-top 2023-02-08 08:54:28,907 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926 2023-02-08 08:54:28,909 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e6e7b00071413982667481a556e71e 2023-02-08 08:54:28,909 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e 2023-02-08 08:54:28,909 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp/cf2/241e192d797a43109b63ce7ad4685b5f as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/241e192d797a43109b63ce7ad4685b5f 2023-02-08 08:54:28,913 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,913 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:28,913 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 of b5a2b4f4020c2cbcbab2bc91399ebe33 into 241e192d797a43109b63ce7ad4685b5f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:28,913 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b5a2b4f4020c2cbcbab2bc91399ebe33: 2023-02-08 08:54:28,913 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33., storeName=b5a2b4f4020c2cbcbab2bc91399ebe33/cf2, priority=9, startTime=1675846468867; duration=0sec 2023-02-08 08:54:28,914 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,915 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:28,915 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:28,916 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(310): Store=d8a3a28ddb9422c08833bf51afe408bf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:28,916 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,916 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 of b5a2b4f4020c2cbcbab2bc91399ebe33 into cd1b38bb2cb747a8a78a9d6c1a43b926(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:28,916 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b5a2b4f4020c2cbcbab2bc91399ebe33: 2023-02-08 08:54:28,917 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33., storeName=b5a2b4f4020c2cbcbab2bc91399ebe33/cf1, priority=9, startTime=1675846468866; duration=0sec 2023-02-08 08:54:28,917 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2 2023-02-08 08:54:28,917 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,917 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2 2023-02-08 08:54:28,917 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8a3a28ddb9422c08833bf51afe408bf columnFamilyName cf2 2023-02-08 08:54:28,925 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-top 2023-02-08 08:54:28,927 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:28,928 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-top 2023-02-08 08:54:28,931 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-top 2023-02-08 08:54:28,938 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-top 2023-02-08 08:54:28,954 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:28,954 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:28,969 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:28,970 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:28,974 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,975 DEBUG [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:28,975 INFO [StoreOpener-d8a3a28ddb9422c08833bf51afe408bf-1] regionserver.HStore(310): Store=d8a3a28ddb9422c08833bf51afe408bf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:28,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:28,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:28,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d8a3a28ddb9422c08833bf51afe408bf; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61867845, jitterRate=-0.07809726893901825}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:28,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d8a3a28ddb9422c08833bf51afe408bf: Running coprocessor pre-open hook at 1675846468870Writing region info on filesystem at 1675846468870Initializing all the Stores at 1675846468871 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846468871Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846468872 (+1 ms)Cleaning up temporary data from old regions at 1675846468977 (+105 ms)Cleaning up detritus from prior splits at 1675846468977Running coprocessor post-open hooks at 1675846468980 (+3 ms)Region opened successfully at 1675846468980 2023-02-08 08:54:28,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf., pid=78, masterSystemTime=1675846468789 2023-02-08 08:54:28,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:28,982 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:28,982 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:54:28,982 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,982 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): d8a3a28ddb9422c08833bf51afe408bf/cf2 is initiating minor compaction (all files) 2023-02-08 08:54:28,982 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,983 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d8a3a28ddb9422c08833bf51afe408bf/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,983 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): d8a3a28ddb9422c08833bf51afe408bf/cf1 is initiating minor compaction (all files) 2023-02-08 08:54:28,983 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d8a3a28ddb9422c08833bf51afe408bf/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,983 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/.tmp, totalSize=34.8 K 2023-02-08 08:54:28,983 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/.tmp, totalSize=34.8 K 2023-02-08 08:54:28,984 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846466340 2023-02-08 08:54:28,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:28,984 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846466340 2023-02-08 08:54:28,985 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846466523 2023-02-08 08:54:28,985 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846466523 2023-02-08 08:54:28,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=d8a3a28ddb9422c08833bf51afe408bf, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:28,985 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846466708 2023-02-08 08:54:28,985 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846468985"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846468985"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846468985"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846468985"}]},"ts":"1675846468985"} 2023-02-08 08:54:28,985 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846466708 2023-02-08 08:54:28,986 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675846467461 2023-02-08 08:54:28,986 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675846467461 2023-02-08 08:54:28,986 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,986 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,987 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,987 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,987 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,987 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 08:54:28,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-08 08:54:28,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure d8a3a28ddb9422c08833bf51afe408bf, server=jenkins-hbase4.apache.org,44651,1675846441425 in 346 msec 2023-02-08 08:54:28,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-08 08:54:28,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, ASSIGN in 508 msec 2023-02-08 08:54:28,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf in 1.0100 sec 2023-02-08 08:54:29,013 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): d8a3a28ddb9422c08833bf51afe408bf#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:29,015 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): d8a3a28ddb9422c08833bf51afe408bf#cf2#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:29,015 DEBUG [Listener at localhost/33105] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:54:29,026 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53348, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:54:29,026 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.HStore(1912): b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 is initiating major compaction (all files) 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@243ae813; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.HStore(1912): b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 is initiating major compaction (all files) 2023-02-08 08:54:29,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@15c95206; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 08:54:29,030 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. because compaction request was cancelled 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:54:29,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. because compaction request was cancelled 2023-02-08 08:54:29,032 INFO [Listener at localhost/33105] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 08:54:29,050 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a059aa5bde034e5698c26e9da0209526: NONE, but ROW specified in column family configuration 2023-02-08 08:54:29,050 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a9ace6a23253498781b7b7a18dd78e32: NONE, but ROW specified in column family configuration 2023-02-08 08:54:29,051 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/.tmp/cf1/a059aa5bde034e5698c26e9da0209526 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/a059aa5bde034e5698c26e9da0209526 2023-02-08 08:54:29,051 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/.tmp/cf2/a9ace6a23253498781b7b7a18dd78e32 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/a9ace6a23253498781b7b7a18dd78e32 2023-02-08 08:54:29,056 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a9ace6a23253498781b7b7a18dd78e32: NONE, but ROW specified in column family configuration 2023-02-08 08:54:29,056 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a059aa5bde034e5698c26e9da0209526: NONE, but ROW specified in column family configuration 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d8a3a28ddb9422c08833bf51afe408bf/cf2 of d8a3a28ddb9422c08833bf51afe408bf into a9ace6a23253498781b7b7a18dd78e32(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d8a3a28ddb9422c08833bf51afe408bf/cf1 of d8a3a28ddb9422c08833bf51afe408bf into a059aa5bde034e5698c26e9da0209526(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:29,059 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d8a3a28ddb9422c08833bf51afe408bf: 2023-02-08 08:54:29,059 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d8a3a28ddb9422c08833bf51afe408bf: 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf., storeName=d8a3a28ddb9422c08833bf51afe408bf/cf2, priority=9, startTime=1675846468982; duration=0sec 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf., storeName=d8a3a28ddb9422c08833bf51afe408bf/cf1, priority=9, startTime=1675846468981; duration=0sec 2023-02-08 08:54:29,059 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 08:54:29,059 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/241e192d797a43109b63ce7ad4685b5f] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp, totalSize=6.8 K 2023-02-08 08:54:29,059 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp, totalSize=6.8 K 2023-02-08 08:54:29,060 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 241e192d797a43109b63ce7ad4685b5f, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846466340 2023-02-08 08:54:29,060 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting cd1b38bb2cb747a8a78a9d6c1a43b926, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846466340 2023-02-08 08:54:29,063 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. is not mergeable because it has references 2023-02-08 08:54:29,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. is not mergeable because it has references 2023-02-08 08:54:29,064 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): b5a2b4f4020c2cbcbab2bc91399ebe33#cf2#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:29,065 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): b5a2b4f4020c2cbcbab2bc91399ebe33#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:29,076 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp/cf1/038f9d29401845b78f7fcfb543ad38f0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/038f9d29401845b78f7fcfb543ad38f0 2023-02-08 08:54:29,076 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/.tmp/cf2/16b1d1c754364bcb9c8b07f147499ce1 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/16b1d1c754364bcb9c8b07f147499ce1 2023-02-08 08:54:29,081 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b5a2b4f4020c2cbcbab2bc91399ebe33/cf1 of b5a2b4f4020c2cbcbab2bc91399ebe33 into 038f9d29401845b78f7fcfb543ad38f0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:29,081 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b5a2b4f4020c2cbcbab2bc91399ebe33/cf2 of b5a2b4f4020c2cbcbab2bc91399ebe33 into 16b1d1c754364bcb9c8b07f147499ce1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:29,082 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b5a2b4f4020c2cbcbab2bc91399ebe33: 2023-02-08 08:54:29,082 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b5a2b4f4020c2cbcbab2bc91399ebe33: 2023-02-08 08:54:29,083 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675846469028; duration=0sec 2023-02-08 08:54:29,083 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675846469028; duration=0sec 2023-02-08 08:54:29,083 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:29,083 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:29,172 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44651] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. is not mergeable because it has references 2023-02-08 08:54:29,173 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44651] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. is not mergeable because it has references 2023-02-08 08:54:29,179 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-08 08:54:29,180 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:29,181 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,181 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,182 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,182 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,182 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,185 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,185 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:54:29,186 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846469186"}]},"ts":"1675846469186"} 2023-02-08 08:54:29,187 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-08 08:54:29,189 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-08 08:54:29,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, UNASSIGN}] 2023-02-08 08:54:29,191 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, UNASSIGN 2023-02-08 08:54:29,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d8a3a28ddb9422c08833bf51afe408bf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:29,191 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846469191"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846469191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846469191"}]},"ts":"1675846469191"} 2023-02-08 08:54:29,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure d8a3a28ddb9422c08833bf51afe408bf, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:29,195 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, UNASSIGN 2023-02-08 08:54:29,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b5a2b4f4020c2cbcbab2bc91399ebe33, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:29,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846469195"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846469195"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846469195"}]},"ts":"1675846469195"} 2023-02-08 08:54:29,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure b5a2b4f4020c2cbcbab2bc91399ebe33, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:29,286 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,286 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:54:29,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:29,344 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d8a3a28ddb9422c08833bf51afe408bf, disabling compactions & flushes 2023-02-08 08:54:29,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:29,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:29,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. after waiting 0 ms 2023-02-08 08:54:29,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:29,345 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c] to archive 2023-02-08 08:54:29,346 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:29,348 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,349 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,350 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,351 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,352 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:29,353 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-58e6e7b00071413982667481a556e71e 2023-02-08 08:54:29,354 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:29,355 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da] to archive 2023-02-08 08:54:29,356 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:29,357 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,358 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,359 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,360 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,361 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:29,363 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:29,363 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:29,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-02-08 08:54:29,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:29,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf. 2023-02-08 08:54:29,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d8a3a28ddb9422c08833bf51afe408bf: Waiting for close lock at 1675846469344Running coprocessor pre-close hooks at 1675846469344Disabling compacts and flushes for region at 1675846469344Disabling writes for close at 1675846469345 (+1 ms)Writing region close event to WAL at 1675846469364 (+19 ms)Running coprocessor post-close hooks at 1675846469367 (+3 ms)Closed at 1675846469368 (+1 ms) 2023-02-08 08:54:29,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:29,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:29,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b5a2b4f4020c2cbcbab2bc91399ebe33, disabling compactions & flushes 2023-02-08 08:54:29,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. after waiting 0 ms 2023-02-08 08:54:29,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d8a3a28ddb9422c08833bf51afe408bf, regionState=CLOSED 2023-02-08 08:54:29,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846469369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846469369"}]},"ts":"1675846469369"} 2023-02-08 08:54:29,370 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79-bottom] to archive 2023-02-08 08:54:29,371 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:29,373 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/8a3e0a59b59a400daa52deca88c5a256.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,376 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6886165c1b6d432c83e5eebaa918c599.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,379 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6b02bdc649164e80806fefaf53db6cb0.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-08 08:54:29,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure d8a3a28ddb9422c08833bf51afe408bf, server=jenkins-hbase4.apache.org,44651,1675846441425 in 179 msec 2023-02-08 08:54:29,380 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-cb7e43086314429da4fb9bd9d56c8eb5 2023-02-08 08:54:29,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d8a3a28ddb9422c08833bf51afe408bf, UNASSIGN in 189 msec 2023-02-08 08:54:29,381 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-05654d915cb74a7e92c06368a1417f4b 2023-02-08 08:54:29,382 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-d96339374f384183924736db96e99229 2023-02-08 08:54:29,384 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/cd1b38bb2cb747a8a78a9d6c1a43b926 2023-02-08 08:54:29,385 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/6311a7895a454195b70cc0f11769af79.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,387 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/241e192d797a43109b63ce7ad4685b5f, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3-bottom] to archive 2023-02-08 08:54:29,388 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:29,389 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/68e2928405ee446db75123f9173e900c.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,390 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/cc685eecccaa491893084f1d6b18b030.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,391 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/f2678c6ee5184285886a0ae01b79ba52.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,392 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-6482253615544c0e8828d0b6eef93caf 2023-02-08 08:54:29,393 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-b8db6e2a357d454fb8d27f95eeeba145 2023-02-08 08:54:29,394 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/testSplitTableRegionDeletedRowsDaughter=2c6d5653f20710b77eab401d7e0f610d-e163002730b04997bbe6792da2c5bb23 2023-02-08 08:54:29,395 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/241e192d797a43109b63ce7ad4685b5f to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/241e192d797a43109b63ce7ad4685b5f 2023-02-08 08:54:29,395 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/6c4124b2a7a94eb88733c56a614b7af3.2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-08 08:54:29,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:29,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33. 2023-02-08 08:54:29,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b5a2b4f4020c2cbcbab2bc91399ebe33: Waiting for close lock at 1675846469369Running coprocessor pre-close hooks at 1675846469369Disabling compacts and flushes for region at 1675846469369Disabling writes for close at 1675846469369Writing region close event to WAL at 1675846469396 (+27 ms)Running coprocessor post-close hooks at 1675846469399 (+3 ms)Closed at 1675846469399 2023-02-08 08:54:29,401 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:29,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b5a2b4f4020c2cbcbab2bc91399ebe33, regionState=CLOSED 2023-02-08 08:54:29,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675846469401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846469401"}]},"ts":"1675846469401"} 2023-02-08 08:54:29,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-08 08:54:29,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure b5a2b4f4020c2cbcbab2bc91399ebe33, server=jenkins-hbase4.apache.org,44651,1675846441425 in 205 msec 2023-02-08 08:54:29,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-08 08:54:29,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b5a2b4f4020c2cbcbab2bc91399ebe33, UNASSIGN in 214 msec 2023-02-08 08:54:29,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846469406"}]},"ts":"1675846469406"} 2023-02-08 08:54:29,407 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-08 08:54:29,408 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-08 08:54:29,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 225 msec 2023-02-08 08:54:29,487 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,487 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:54:29,488 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-08 08:54:29,488 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,488 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,491 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,492 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,492 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,492 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 08:54:29,497 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,497 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:29,497 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:29,499 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/recovered.edits] 2023-02-08 08:54:29,499 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/recovered.edits] 2023-02-08 08:54:29,499 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/recovered.edits] 2023-02-08 08:54:29,501 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/038f9d29401845b78f7fcfb543ad38f0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf1/038f9d29401845b78f7fcfb543ad38f0 2023-02-08 08:54:29,501 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/a059aa5bde034e5698c26e9da0209526 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf1/a059aa5bde034e5698c26e9da0209526 2023-02-08 08:54:29,502 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/05654d915cb74a7e92c06368a1417f4b 2023-02-08 08:54:29,503 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/58e6e7b00071413982667481a556e71e 2023-02-08 08:54:29,504 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/a9ace6a23253498781b7b7a18dd78e32 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/cf2/a9ace6a23253498781b7b7a18dd78e32 2023-02-08 08:54:29,504 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/16b1d1c754364bcb9c8b07f147499ce1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/cf2/16b1d1c754364bcb9c8b07f147499ce1 2023-02-08 08:54:29,505 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6311a7895a454195b70cc0f11769af79 2023-02-08 08:54:29,507 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6886165c1b6d432c83e5eebaa918c599 2023-02-08 08:54:29,508 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/recovered.edits/111.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf/recovered.edits/111.seqid 2023-02-08 08:54:29,508 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/recovered.edits/113.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33/recovered.edits/113.seqid 2023-02-08 08:54:29,508 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:54:29,508 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b5a2b4f4020c2cbcbab2bc91399ebe33 2023-02-08 08:54:29,509 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/6b02bdc649164e80806fefaf53db6cb0 2023-02-08 08:54:29,510 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/8a3e0a59b59a400daa52deca88c5a256 2023-02-08 08:54:29,511 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/c31435c57f5443a79cfedc4f2b1b8f0c 2023-02-08 08:54:29,512 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/cb7e43086314429da4fb9bd9d56c8eb5 2023-02-08 08:54:29,513 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d7d763547fd9464798dbed45d880d7a1 2023-02-08 08:54:29,514 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf1/d96339374f384183924736db96e99229 2023-02-08 08:54:29,516 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6364e3599ba14b148be20082f26c8f2a 2023-02-08 08:54:29,517 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6482253615544c0e8828d0b6eef93caf 2023-02-08 08:54:29,519 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/68e2928405ee446db75123f9173e900c 2023-02-08 08:54:29,520 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/6c4124b2a7a94eb88733c56a614b7af3 2023-02-08 08:54:29,521 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/a369aae42c7546b9aec167ceafa048e4 2023-02-08 08:54:29,522 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/b8db6e2a357d454fb8d27f95eeeba145 2023-02-08 08:54:29,523 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/cc685eecccaa491893084f1d6b18b030 2023-02-08 08:54:29,524 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/d02f10b2ec3a4c7a8a8badcbfc79a1da 2023-02-08 08:54:29,525 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/e163002730b04997bbe6792da2c5bb23 2023-02-08 08:54:29,526 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/cf2/f2678c6ee5184285886a0ae01b79ba52 2023-02-08 08:54:29,528 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/recovered.edits/106.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d/recovered.edits/106.seqid 2023-02-08 08:54:29,529 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c6d5653f20710b77eab401d7e0f610d 2023-02-08 08:54:29,529 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 08:54:29,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,533 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-08 08:54:29,535 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-08 08:54:29,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-08 08:54:29,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846469536"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:29,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846469536"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:29,536 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846469536"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:29,537 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:54:29,537 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2c6d5653f20710b77eab401d7e0f610d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675846465836.2c6d5653f20710b77eab401d7e0f610d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b5a2b4f4020c2cbcbab2bc91399ebe33, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675846467981.b5a2b4f4020c2cbcbab2bc91399ebe33.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => d8a3a28ddb9422c08833bf51afe408bf, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675846467981.d8a3a28ddb9422c08833bf51afe408bf.', STARTKEY => '60', ENDKEY => ''}] 2023-02-08 08:54:29,537 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-08 08:54:29,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846469538"}]},"ts":"9223372036854775807"} 2023-02-08 08:54:29,539 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-08 08:54:29,541 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:54:29,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-02-08 08:54:29,593 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,593 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 08:54:29,594 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-08 08:54:29,609 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=410 (was 406) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:50228 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:45662 [Waiting for operation #41] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:58590 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:58864 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:49170 [Waiting for operation #44] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:51378 [Waiting for operation #43] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:60304 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:50510 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:51980 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:38366 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:58828 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:60120 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:44416 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:51934 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c175139-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:58628 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1156508785_15 at /127.0.0.1:50524 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=872 (was 870) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=328 (was 287) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4235 (was 4312) 2023-02-08 08:54:29,623 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=410, OpenFileDescriptor=872, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=168, AvailableMemoryMB=4234 2023-02-08 08:54:29,624 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:54:29,624 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,624 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:29,626 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:54:29,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:54:29,627 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:54:29,629 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:29,629 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:29,631 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,631 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a empty. 2023-02-08 08:54:29,632 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,632 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 08:54:29,641 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:29,642 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 42130720bfc979feffebd5919715188a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:54:29,650 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:29,650 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 42130720bfc979feffebd5919715188a, disabling compactions & flushes 2023-02-08 08:54:29,650 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,650 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,650 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. after waiting 0 ms 2023-02-08 08:54:29,650 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,651 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,651 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 42130720bfc979feffebd5919715188a: Waiting for close lock at 1675846469650Disabling compacts and flushes for region at 1675846469650Disabling writes for close at 1675846469650Writing region close event to WAL at 1675846469651 (+1 ms)Closed at 1675846469651 2023-02-08 08:54:29,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:29,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846469653"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846469653"}]},"ts":"1675846469653"} 2023-02-08 08:54:29,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:29,655 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:29,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846469655"}]},"ts":"1675846469655"} 2023-02-08 08:54:29,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-08 08:54:29,659 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:29,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:29,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:29,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:29,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:54:29,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, ASSIGN}] 2023-02-08 08:54:29,661 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, ASSIGN 2023-02-08 08:54:29,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:54:29,812 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:29,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:29,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846469815"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846469815"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846469815"}]},"ts":"1675846469815"} 2023-02-08 08:54:29,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:29,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 42130720bfc979feffebd5919715188a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:29,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:29,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:29,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,972 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,974 DEBUG [StoreOpener-42130720bfc979feffebd5919715188a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:54:29,974 DEBUG [StoreOpener-42130720bfc979feffebd5919715188a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:54:29,974 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42130720bfc979feffebd5919715188a columnFamilyName cf1 2023-02-08 08:54:29,974 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] regionserver.HStore(310): Store=42130720bfc979feffebd5919715188a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:29,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:29,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:29,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 42130720bfc979feffebd5919715188a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65798559, jitterRate=-0.019525066018104553}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:29,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 42130720bfc979feffebd5919715188a: Running coprocessor pre-open hook at 1675846469971Writing region info on filesystem at 1675846469971Initializing all the Stores at 1675846469972 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846469972Cleaning up temporary data from old regions at 1675846469976 (+4 ms)Cleaning up detritus from prior splits at 1675846469976Running coprocessor post-open hooks at 1675846469979 (+3 ms)Region opened successfully at 1675846469979 2023-02-08 08:54:29,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a., pid=87, masterSystemTime=1675846469968 2023-02-08 08:54:29,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:29,981 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:29,982 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846469981"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846469981"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846469981"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846469981"}]},"ts":"1675846469981"} 2023-02-08 08:54:29,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-08 08:54:29,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504 in 167 msec 2023-02-08 08:54:29,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-08 08:54:29,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, ASSIGN in 324 msec 2023-02-08 08:54:29,987 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:29,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846469987"}]},"ts":"1675846469987"} 2023-02-08 08:54:29,988 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-08 08:54:29,994 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:29,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 368 msec 2023-02-08 08:54:30,128 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:30,128 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:30,130 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:30,130 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:30,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 08:54:30,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:54:30,157 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 08:54:30,157 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:30,157 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 08:54:30,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-08 08:54:30,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, REOPEN/MOVE}] 2023-02-08 08:54:30,173 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, REOPEN/MOVE 2023-02-08 08:54:30,174 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:30,174 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846470174"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846470174"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846470174"}]},"ts":"1675846470174"} 2023-02-08 08:54:30,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:30,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 42130720bfc979feffebd5919715188a, disabling compactions & flushes 2023-02-08 08:54:30,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. after waiting 0 ms 2023-02-08 08:54:30,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 08:54:30,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:54:30,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 42130720bfc979feffebd5919715188a: Waiting for close lock at 1675846470327Running coprocessor pre-close hooks at 1675846470327Disabling compacts and flushes for region at 1675846470327Disabling writes for close at 1675846470328 (+1 ms)Writing region close event to WAL at 1675846470328Running coprocessor post-close hooks at 1675846470332 (+4 ms)Closed at 1675846470332 2023-02-08 08:54:30,332 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 42130720bfc979feffebd5919715188a to self. 2023-02-08 08:54:30,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=CLOSED 2023-02-08 08:54:30,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846470334"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846470334"}]},"ts":"1675846470334"} 2023-02-08 08:54:30,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-08 08:54:30,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504 in 160 msec 2023-02-08 08:54:30,337 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=true 2023-02-08 08:54:30,488 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:54:30,488 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:30,488 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846470488"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846470488"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846470488"}]},"ts":"1675846470488"} 2023-02-08 08:54:30,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:54:30,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 42130720bfc979feffebd5919715188a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:30,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:30,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:30,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,646 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,647 DEBUG [StoreOpener-42130720bfc979feffebd5919715188a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:54:30,647 DEBUG [StoreOpener-42130720bfc979feffebd5919715188a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:54:30,647 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 42130720bfc979feffebd5919715188a columnFamilyName cf1 2023-02-08 08:54:30,648 INFO [StoreOpener-42130720bfc979feffebd5919715188a-1] regionserver.HStore(310): Store=42130720bfc979feffebd5919715188a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:30,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 42130720bfc979feffebd5919715188a 2023-02-08 08:54:30,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 42130720bfc979feffebd5919715188a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60040176, jitterRate=-0.1053316593170166}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:30,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 42130720bfc979feffebd5919715188a: Running coprocessor pre-open hook at 1675846470645Writing region info on filesystem at 1675846470645Initializing all the Stores at 1675846470646 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846470646Cleaning up temporary data from old regions at 1675846470650 (+4 ms)Cleaning up detritus from prior splits at 1675846470650Running coprocessor post-open hooks at 1675846470652 (+2 ms)Region opened successfully at 1675846470652 2023-02-08 08:54:30,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a., pid=92, masterSystemTime=1675846470641 2023-02-08 08:54:30,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:54:30,655 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:54:30,655 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846470655"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846470655"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846470655"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846470655"}]},"ts":"1675846470655"} 2023-02-08 08:54:30,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-08 08:54:30,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504 in 166 msec 2023-02-08 08:54:30,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-08 08:54:30,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, REOPEN/MOVE in 486 msec 2023-02-08 08:54:30,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-08 08:54:30,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-02-08 08:54:30,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, ASSIGN}] 2023-02-08 08:54:30,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, ASSIGN 2023-02-08 08:54:30,667 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 08:54:30,817 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:54:30,818 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:54:30,818 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:54:30,818 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:54:30,818 DEBUG [jenkins-hbase4:33127] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-08 08:54:30,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=786f8bf894e79e4fdc03875423343e48, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:54:30,820 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846470820"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675846470820"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675846470820"}]},"ts":"1675846470820"} 2023-02-08 08:54:30,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 786f8bf894e79e4fdc03875423343e48, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:54:30,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:54:30,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 786f8bf894e79e4fdc03875423343e48, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-08 08:54:30,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:54:30,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:54:30,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:30,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:54:30,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:54:30,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:31,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:32,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,009 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-08 08:54:33,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,687 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:33,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:33,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:34,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:35,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:36,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:37,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:38,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:39,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:40,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:41,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:42,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:43,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:44,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:45,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:46,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:47,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:48,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:49,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:50,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:51,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:52,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:53,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:54,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:55,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:56,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:57,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:58,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:54:59,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:00,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:01,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,884 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 08:55:02,902 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 08:55:02,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:02,957 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 08:55:03,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:03,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:04,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:05,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:06,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:07,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:08,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:09,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:10,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:11,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:12,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:13,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:14,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:15,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:16,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:17,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:18,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:19,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:20,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:21,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:22,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:23,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:24,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:25,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:26,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:27,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:28,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:29,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,253 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,253 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,256 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,256 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,357 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,357 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,559 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,559 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,860 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,860 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:30,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,361 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:31,362 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:31,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:31,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:31,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,364 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:32,365 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:32,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:32,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:32,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:32,368 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 42130720bfc979feffebd5919715188a, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:32,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca exec-time=6 msec 2023-02-08 08:55:32,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 08:55:32,703 INFO [StoreOpener-786f8bf894e79e4fdc03875423343e48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:55:32,707 DEBUG [StoreOpener-786f8bf894e79e4fdc03875423343e48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:55:32,707 DEBUG [StoreOpener-786f8bf894e79e4fdc03875423343e48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1 2023-02-08 08:55:32,708 INFO [StoreOpener-786f8bf894e79e4fdc03875423343e48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 786f8bf894e79e4fdc03875423343e48 columnFamilyName cf1 2023-02-08 08:55:32,709 INFO [StoreOpener-786f8bf894e79e4fdc03875423343e48-1] regionserver.HStore(310): Store=786f8bf894e79e4fdc03875423343e48/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:32,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 786f8bf894e79e4fdc03875423343e48; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60745837, jitterRate=-0.09481649100780487}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:55:32,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 786f8bf894e79e4fdc03875423343e48: Running coprocessor pre-open hook at 1675846470979Initializing all the Stores at 1675846532702 (+61723 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846532702Running coprocessor post-open hooks at 1675846532710 (+8 ms)Region opened successfully at 1675846532710 2023-02-08 08:55:32,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48., pid=94, masterSystemTime=1675846470976 2023-02-08 08:55:32,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:32,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:32,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=786f8bf894e79e4fdc03875423343e48, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:32,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846532712"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675846532712"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675846532712"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675846532712"}]},"ts":"1675846532712"} 2023-02-08 08:55:32,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-08 08:55:32,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 786f8bf894e79e4fdc03875423343e48, server=jenkins-hbase4.apache.org,44651,1675846441425 in 1 mins, 1.892 sec 2023-02-08 08:55:32,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-08 08:55:32,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, ASSIGN in 1 mins, 2.05 sec 2023-02-08 08:55:32,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.578 sec 2023-02-08 08:55:32,719 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675846470134","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62584,"client":"172.31.14.131:51158","queuetimems":0,"class":"HMaster"} 2023-02-08 08:55:32,719 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-08 08:55:34,616 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up 2023-02-08 08:55:34,617 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:34,617 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,617 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,619 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,619 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,619 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,622 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,622 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:34,623 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846534623"}]},"ts":"1675846534623"} 2023-02-08 08:55:34,624 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-08 08:55:34,628 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-08 08:55:34,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, UNASSIGN}] 2023-02-08 08:55:34,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, UNASSIGN 2023-02-08 08:55:34,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=786f8bf894e79e4fdc03875423343e48, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:34,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846534630"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675846534630"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675846534630"}]},"ts":"1675846534630"} 2023-02-08 08:55:34,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 786f8bf894e79e4fdc03875423343e48, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:34,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, UNASSIGN 2023-02-08 08:55:34,633 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:34,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846534633"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846534633"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846534633"}]},"ts":"1675846534633"} 2023-02-08 08:55:34,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:34,723 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,723 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:34,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 786f8bf894e79e4fdc03875423343e48, disabling compactions & flushes 2023-02-08 08:55:34,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. after waiting 0 ms 2023-02-08 08:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:34,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48. 2023-02-08 08:55:34,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 786f8bf894e79e4fdc03875423343e48: Waiting for close lock at 1675846534783Running coprocessor pre-close hooks at 1675846534783Disabling compacts and flushes for region at 1675846534783Disabling writes for close at 1675846534784 (+1 ms)Writing region close event to WAL at 1675846534784Running coprocessor post-close hooks at 1675846534784Closed at 1675846534785 (+1 ms) 2023-02-08 08:55:34,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 42130720bfc979feffebd5919715188a 2023-02-08 08:55:34,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 42130720bfc979feffebd5919715188a, disabling compactions & flushes 2023-02-08 08:55:34,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:55:34,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:55:34,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. after waiting 0 ms 2023-02-08 08:55:34,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:55:34,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 786f8bf894e79e4fdc03875423343e48 2023-02-08 08:55:34,788 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=786f8bf894e79e4fdc03875423343e48, regionState=CLOSED 2023-02-08 08:55:34,788 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846534788"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675846534788"}]},"ts":"1675846534788"} 2023-02-08 08:55:34,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 08:55:34,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-08 08:55:34,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:34,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 786f8bf894e79e4fdc03875423343e48, server=jenkins-hbase4.apache.org,44651,1675846441425 in 159 msec 2023-02-08 08:55:34,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a. 2023-02-08 08:55:34,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 42130720bfc979feffebd5919715188a: Waiting for close lock at 1675846534786Running coprocessor pre-close hooks at 1675846534786Disabling compacts and flushes for region at 1675846534786Disabling writes for close at 1675846534787 (+1 ms)Writing region close event to WAL at 1675846534787Running coprocessor post-close hooks at 1675846534792 (+5 ms)Closed at 1675846534792 2023-02-08 08:55:34,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48, UNASSIGN in 163 msec 2023-02-08 08:55:34,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 42130720bfc979feffebd5919715188a 2023-02-08 08:55:34,794 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=42130720bfc979feffebd5919715188a, regionState=CLOSED 2023-02-08 08:55:34,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675846534794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846534794"}]},"ts":"1675846534794"} 2023-02-08 08:55:34,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-08 08:55:34,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 42130720bfc979feffebd5919715188a, server=jenkins-hbase4.apache.org,39055,1675846441504 in 161 msec 2023-02-08 08:55:34,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-08 08:55:34,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=42130720bfc979feffebd5919715188a, UNASSIGN in 168 msec 2023-02-08 08:55:34,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846534800"}]},"ts":"1675846534800"} 2023-02-08 08:55:34,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-08 08:55:34,803 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-08 08:55:34,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 184 msec 2023-02-08 08:55:34,924 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,924 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:34,925 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-08 08:55:34,925 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,925 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,928 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,929 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,929 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 08:55:34,935 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:55:34,936 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits] 2023-02-08 08:55:34,940 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits/7.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a/recovered.edits/7.seqid 2023-02-08 08:55:34,941 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/42130720bfc979feffebd5919715188a 2023-02-08 08:55:34,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 08:55:34,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,945 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-08 08:55:34,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-08 08:55:34,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-08 08:55:34,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846534950"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:34,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846534950"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:34,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-08 08:55:34,952 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 42130720bfc979feffebd5919715188a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626.42130720bfc979feffebd5919715188a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 786f8bf894e79e4fdc03875423343e48, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675846469626_0001.786f8bf894e79e4fdc03875423343e48.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-08 08:55:34,952 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-08 08:55:34,952 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846534952"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:34,954 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-08 08:55:34,956 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:34,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 30 msec 2023-02-08 08:55:35,029 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,030 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 08:55:35,031 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-08 08:55:35,044 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=344 (was 410), OpenFileDescriptor=655 (was 872), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 328), ProcessCount=168 (was 168), AvailableMemoryMB=4043 (was 4234) 2023-02-08 08:55:35,056 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=345, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=136, ProcessCount=168, AvailableMemoryMB=4042 2023-02-08 08:55:35,057 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:35,057 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,057 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,059 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:55:35,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:55:35,061 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:35,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:35,063 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:35,065 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,065 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae empty. 2023-02-08 08:55:35,065 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,066 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 08:55:35,076 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:35,077 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 44cfa83816aa4855c5a36094c2b511ae, NAME => 'testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:55:35,090 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:35,091 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 44cfa83816aa4855c5a36094c2b511ae, disabling compactions & flushes 2023-02-08 08:55:35,091 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,091 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,091 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. after waiting 0 ms 2023-02-08 08:55:35,091 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,091 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,091 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 44cfa83816aa4855c5a36094c2b511ae: Waiting for close lock at 1675846535091Disabling compacts and flushes for region at 1675846535091Disabling writes for close at 1675846535091Writing region close event to WAL at 1675846535091Closed at 1675846535091 2023-02-08 08:55:35,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:35,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675846535094"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846535094"}]},"ts":"1675846535094"} 2023-02-08 08:55:35,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:35,096 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:35,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846535096"}]},"ts":"1675846535096"} 2023-02-08 08:55:35,098 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-08 08:55:35,102 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:55:35,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:55:35,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:55:35,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:55:35,103 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:55:35,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, ASSIGN}] 2023-02-08 08:55:35,104 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, ASSIGN 2023-02-08 08:55:35,105 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:55:35,255 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:55:35,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=44cfa83816aa4855c5a36094c2b511ae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675846535256"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846535256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846535256"}]},"ts":"1675846535256"} 2023-02-08 08:55:35,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 44cfa83816aa4855c5a36094c2b511ae, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:35,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 44cfa83816aa4855c5a36094c2b511ae, NAME => 'testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:35,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:35,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:35,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,415 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,416 DEBUG [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1 2023-02-08 08:55:35,416 DEBUG [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1 2023-02-08 08:55:35,416 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 44cfa83816aa4855c5a36094c2b511ae columnFamilyName cf1 2023-02-08 08:55:35,417 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] regionserver.HStore(310): Store=44cfa83816aa4855c5a36094c2b511ae/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:35,417 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,418 DEBUG [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2 2023-02-08 08:55:35,418 DEBUG [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2 2023-02-08 08:55:35,418 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 44cfa83816aa4855c5a36094c2b511ae columnFamilyName cf2 2023-02-08 08:55:35,419 INFO [StoreOpener-44cfa83816aa4855c5a36094c2b511ae-1] regionserver.HStore(310): Store=44cfa83816aa4855c5a36094c2b511ae/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:35,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:35,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:35,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:35,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 44cfa83816aa4855c5a36094c2b511ae; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67983740, jitterRate=0.013036668300628662}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:35,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 44cfa83816aa4855c5a36094c2b511ae: Running coprocessor pre-open hook at 1675846535414Writing region info on filesystem at 1675846535414Initializing all the Stores at 1675846535414Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846535414Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846535415 (+1 ms)Cleaning up temporary data from old regions at 1675846535421 (+6 ms)Cleaning up detritus from prior splits at 1675846535421Running coprocessor post-open hooks at 1675846535424 (+3 ms)Region opened successfully at 1675846535424 2023-02-08 08:55:35,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae., pid=104, masterSystemTime=1675846535410 2023-02-08 08:55:35,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=44cfa83816aa4855c5a36094c2b511ae, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675846535426"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846535426"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846535426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846535426"}]},"ts":"1675846535426"} 2023-02-08 08:55:35,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-08 08:55:35,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 44cfa83816aa4855c5a36094c2b511ae, server=jenkins-hbase4.apache.org,39055,1675846441504 in 170 msec 2023-02-08 08:55:35,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-08 08:55:35,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, ASSIGN in 326 msec 2023-02-08 08:55:35,431 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:35,431 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846535431"}]},"ts":"1675846535431"} 2023-02-08 08:55:35,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-08 08:55:35,435 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:35,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 375 msec 2023-02-08 08:55:35,568 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,568 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:35,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:35,570 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:35,570 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,570 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:35,570 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:35,571 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,571 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,572 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:35,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,574 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:35,574 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,574 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,574 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:35,575 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:55:35,575 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,576 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,576 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,576 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,577 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,577 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,585 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/4db7e213493e4ad09b258d166487dc86 2023-02-08 08:55:35,597 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/07d23c8a04e048829fa7296a6962b04e 2023-02-08 08:55:35,601 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/4db7e213493e4ad09b258d166487dc86 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/4db7e213493e4ad09b258d166487dc86 2023-02-08 08:55:35,605 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/4db7e213493e4ad09b258d166487dc86, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:35,606 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/07d23c8a04e048829fa7296a6962b04e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/07d23c8a04e048829fa7296a6962b04e 2023-02-08 08:55:35,611 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/07d23c8a04e048829fa7296a6962b04e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:35,612 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 37ms, sequenceid=6, compaction requested=false 2023-02-08 08:55:35,612 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-08 08:55:35,613 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:35,613 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,613 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:35,613 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:35,613 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,613 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,613 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,616 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,616 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,616 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,616 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b44129b[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:35,619 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:35,619 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,620 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,620 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:35,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,620 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,621 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:35,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:35,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,621 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:35,622 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,622 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,622 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,622 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,623 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,623 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,623 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,625 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,625 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,625 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:35,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:35,627 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:35,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:35,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:35,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:35,628 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,629 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,630 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,630 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,630 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,632 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,632 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,632 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,635 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:35,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:35,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,635 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,636 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,636 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:35,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,636 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:35,636 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,636 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,637 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,637 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,637 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,637 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,637 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,736 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:35,737 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,737 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:35,753 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,753 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:35,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:35,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:35,755 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:35,755 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,755 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:35,755 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:35,756 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,756 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,757 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,757 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,759 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:35,760 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:35,760 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,760 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:35,760 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,773 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/0409ae208faa42a080d99d7069d7f1d4 2023-02-08 08:55:35,783 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/cf6c68f5eace4d45880072eb5a9b0945 2023-02-08 08:55:35,786 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/0409ae208faa42a080d99d7069d7f1d4 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4 2023-02-08 08:55:35,789 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:35,790 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/cf6c68f5eace4d45880072eb5a9b0945 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945 2023-02-08 08:55:35,793 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:35,794 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 34ms, sequenceid=19, compaction requested=false 2023-02-08 08:55:35,794 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:35,794 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,794 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:35,794 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:35,794 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,794 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,794 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,796 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,796 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:35,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3309a915[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:35,798 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:35,798 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,799 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:35,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,802 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,803 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,803 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:35,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:35,805 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:35,805 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:35,805 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:35,805 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:35,805 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,807 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,807 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,808 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,808 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,809 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,810 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,810 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:35,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:35,813 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:35,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,813 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,814 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,814 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,814 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,814 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,914 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:35,914 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,914 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:35,925 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,925 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:35,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:35,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:35,927 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:35,927 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,927 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:35,928 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:35,928 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,928 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,930 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,930 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,930 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,930 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,931 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:35,933 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:35,933 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,932 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:35,933 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:35,933 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:35,933 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:35,933 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,933 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,933 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:35,934 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,934 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,934 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,935 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,935 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,935 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,941 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/7e43829bd6fc49eca5ca952857871758 2023-02-08 08:55:35,954 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/77d0c3ea6c384fd58d91572f0517e729 2023-02-08 08:55:35,958 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/7e43829bd6fc49eca5ca952857871758 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/7e43829bd6fc49eca5ca952857871758 2023-02-08 08:55:35,961 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/7e43829bd6fc49eca5ca952857871758, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:35,962 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/77d0c3ea6c384fd58d91572f0517e729 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/77d0c3ea6c384fd58d91572f0517e729 2023-02-08 08:55:35,966 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/77d0c3ea6c384fd58d91572f0517e729, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:35,967 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 34ms, sequenceid=32, compaction requested=true 2023-02-08 08:55:35,967 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:35,967 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:35,969 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,969 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,969 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,969 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:35,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@563f5fd5[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:35,971 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:35,971 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,973 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:35,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,973 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,973 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:35,973 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:35,975 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,975 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,975 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,975 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:35,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:35,978 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:35,978 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:35,979 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:35,979 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:35,979 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,981 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,981 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:35,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:35,982 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:35,983 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:35,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:35,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,984 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,984 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:35,985 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,991 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:35,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:35,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:35,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:35,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,992 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:35,991 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:35,992 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:35,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,992 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,992 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:35,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:35,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,992 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:35,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:35,993 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:35,993 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:35,993 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,092 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:36,093 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,093 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:36,105 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,105 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:36,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:36,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:36,107 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:36,107 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,107 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:36,107 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:36,107 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,107 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,108 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,108 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,108 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,108 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,109 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,109 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,110 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,111 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,111 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:36,111 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:36,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,111 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,112 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,114 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,114 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,114 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,114 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,114 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,120 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/e0a34fc07b4240ea8d0738115de11e84 2023-02-08 08:55:36,131 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/6e0357a15fa54444b2850ec4d4cf59b7 2023-02-08 08:55:36,134 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/e0a34fc07b4240ea8d0738115de11e84 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/e0a34fc07b4240ea8d0738115de11e84 2023-02-08 08:55:36,138 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/e0a34fc07b4240ea8d0738115de11e84, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:36,138 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/6e0357a15fa54444b2850ec4d4cf59b7 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/6e0357a15fa54444b2850ec4d4cf59b7 2023-02-08 08:55:36,143 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/6e0357a15fa54444b2850ec4d4cf59b7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:36,144 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 33ms, sequenceid=45, compaction requested=true 2023-02-08 08:55:36,144 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:36,144 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,144 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:36,144 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:36,144 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,144 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,144 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:36,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fab0d50[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:36,149 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:36,149 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:36,150 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:36,150 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:36,152 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,152 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,153 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,153 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,153 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:36,156 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:36,156 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:36,156 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:36,156 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:36,156 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,159 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,158 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,159 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,161 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,161 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,162 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:36,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,164 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,164 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,165 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,166 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,265 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:36,265 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,266 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:36,278 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,278 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:36,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:36,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:36,280 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:36,280 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,280 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:36,280 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:36,281 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,281 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,282 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,282 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,282 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,282 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,291 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,291 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,291 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,293 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,293 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,293 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,294 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,295 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,295 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:36,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,300 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,300 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,301 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,301 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,301 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:36,301 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,301 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,302 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,303 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,303 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,312 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/9864a33547c64f60be1ea82840c9ee98 2023-02-08 08:55:36,323 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/c04bddc539274e34909bd1d433f7b93e 2023-02-08 08:55:36,327 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/9864a33547c64f60be1ea82840c9ee98 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/9864a33547c64f60be1ea82840c9ee98 2023-02-08 08:55:36,330 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/9864a33547c64f60be1ea82840c9ee98, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:36,331 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/c04bddc539274e34909bd1d433f7b93e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c04bddc539274e34909bd1d433f7b93e 2023-02-08 08:55:36,335 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c04bddc539274e34909bd1d433f7b93e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:36,335 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 34ms, sequenceid=58, compaction requested=true 2023-02-08 08:55:36,335 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:36,336 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,336 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:36,336 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:36,336 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,336 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,336 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,338 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,338 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,339 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,339 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:36,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24d006f4[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:36,341 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:36,341 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,343 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,343 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:36,343 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,347 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,350 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,350 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,350 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,350 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:36,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:36,354 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:36,354 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:36,354 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:36,354 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:36,354 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:36,355 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,355 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,355 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,356 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,356 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,356 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,356 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,357 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,358 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,358 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,359 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:36,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:36,361 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,361 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,362 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,462 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:36,462 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,462 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:36,474 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,474 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:36,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:36,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 08:55:36,476 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 08:55:36,476 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,476 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 08:55:36,476 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 08:55:36,477 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,477 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,479 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,479 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,479 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,479 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,479 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,480 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:36,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 08:55:36,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 08:55:36,481 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,481 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. started... 2023-02-08 08:55:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,482 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,482 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,490 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/5cee070e1a264629810312ce133259e6 2023-02-08 08:55:36,500 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/c6fc41833dce47f6aa1874fee2e4a09d 2023-02-08 08:55:36,504 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/5cee070e1a264629810312ce133259e6 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/5cee070e1a264629810312ce133259e6 2023-02-08 08:55:36,507 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/5cee070e1a264629810312ce133259e6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:36,507 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/c6fc41833dce47f6aa1874fee2e4a09d as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c6fc41833dce47f6aa1874fee2e4a09d 2023-02-08 08:55:36,510 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c6fc41833dce47f6aa1874fee2e4a09d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:36,511 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 29ms, sequenceid=71, compaction requested=true 2023-02-08 08:55:36,511 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:36,511 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,511 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:36,511 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:36,511 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,511 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 08:55:36,511 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 08:55:36,512 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,512 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,513 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,513 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 08:55:36,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5aef427a[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:36,514 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 08:55:36,514 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,516 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,517 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:36,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 08:55:36,517 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:36,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,518 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 08:55:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,519 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 08:55:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:36,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 08:55:36,522 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 08:55:36,522 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:36,522 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:36,522 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 08:55:36,522 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,525 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:36,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,526 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,526 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:36,526 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:36,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 08:55:36,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,528 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,530 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 08:55:36,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:36,531 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:36,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:36,531 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 08:55:36,532 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 08:55:36,632 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:36,632 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,632 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:36,639 INFO [Listener at localhost/33105] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,640 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.StoreUtils(129): cannot split hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4 because midkey is the same as first or last row 2023-02-08 08:55:36,640 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.StoreUtils(129): cannot split hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945 because midkey is the same as first or last row 2023-02-08 08:55:36,640 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HRegion(2744): Flushing 44cfa83816aa4855c5a36094c2b511ae 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:55:36,649 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/ee95517ac2154808988acb1615498af6 2023-02-08 08:55:36,659 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/52bb41d114d048a08c976d1db9299159 2023-02-08 08:55:36,663 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf1/ee95517ac2154808988acb1615498af6 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/ee95517ac2154808988acb1615498af6 2023-02-08 08:55:36,666 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/ee95517ac2154808988acb1615498af6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:36,667 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/.tmp/cf2/52bb41d114d048a08c976d1db9299159 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/52bb41d114d048a08c976d1db9299159 2023-02-08 08:55:36,670 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/52bb41d114d048a08c976d1db9299159, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:36,671 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 44cfa83816aa4855c5a36094c2b511ae in 31ms, sequenceid=82, compaction requested=true 2023-02-08 08:55:36,672 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.HRegion(2445): Flush status journal for 44cfa83816aa4855c5a36094c2b511ae: 2023-02-08 08:55:36,672 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.StoreUtils(129): cannot split hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4 because midkey is the same as first or last row 2023-02-08 08:55:36,672 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39055] regionserver.StoreUtils(129): cannot split hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945 because midkey is the same as first or last row 2023-02-08 08:55:36,672 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,672 DEBUG [Listener at localhost/33105] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-08 08:55:36,672 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:36,672 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,672 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,674 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,674 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,674 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-08 08:55:36,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-08 08:55:36,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:36,677 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,677 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:36,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846536677"}]},"ts":"1675846536677"} 2023-02-08 08:55:36,678 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-08 08:55:36,681 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-08 08:55:36,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, UNASSIGN}] 2023-02-08 08:55:36,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, UNASSIGN 2023-02-08 08:55:36,682 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=44cfa83816aa4855c5a36094c2b511ae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:36,682 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675846536682"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846536682"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846536682"}]},"ts":"1675846536682"} 2023-02-08 08:55:36,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 44cfa83816aa4855c5a36094c2b511ae, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:36,778 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,778 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:36,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:36,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 44cfa83816aa4855c5a36094c2b511ae, disabling compactions & flushes 2023-02-08 08:55:36,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. after waiting 0 ms 2023-02-08 08:55:36,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:55:36,839 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:36,839 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae. 2023-02-08 08:55:36,840 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 44cfa83816aa4855c5a36094c2b511ae: Waiting for close lock at 1675846536835Running coprocessor pre-close hooks at 1675846536835Disabling compacts and flushes for region at 1675846536835Disabling writes for close at 1675846536835Writing region close event to WAL at 1675846536836 (+1 ms)Running coprocessor post-close hooks at 1675846536839 (+3 ms)Closed at 1675846536839 2023-02-08 08:55:36,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:36,841 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=44cfa83816aa4855c5a36094c2b511ae, regionState=CLOSED 2023-02-08 08:55:36,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675846536841"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846536841"}]},"ts":"1675846536841"} 2023-02-08 08:55:36,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-08 08:55:36,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 44cfa83816aa4855c5a36094c2b511ae, server=jenkins-hbase4.apache.org,39055,1675846441504 in 159 msec 2023-02-08 08:55:36,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-08 08:55:36,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=44cfa83816aa4855c5a36094c2b511ae, UNASSIGN in 163 msec 2023-02-08 08:55:36,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846536846"}]},"ts":"1675846536846"} 2023-02-08 08:55:36,847 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-08 08:55:36,848 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-08 08:55:36,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 174 msec 2023-02-08 08:55:36,979 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,979 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:36,980 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-08 08:55:36,980 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,980 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-08 08:55:36,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:36,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:36,983 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,984 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:36,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 08:55:36,987 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:36,988 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/recovered.edits] 2023-02-08 08:55:36,992 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/0409ae208faa42a080d99d7069d7f1d4 2023-02-08 08:55:36,993 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/4db7e213493e4ad09b258d166487dc86 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/4db7e213493e4ad09b258d166487dc86 2023-02-08 08:55:36,994 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/5cee070e1a264629810312ce133259e6 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/5cee070e1a264629810312ce133259e6 2023-02-08 08:55:36,995 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/7e43829bd6fc49eca5ca952857871758 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/7e43829bd6fc49eca5ca952857871758 2023-02-08 08:55:36,996 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/9864a33547c64f60be1ea82840c9ee98 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/9864a33547c64f60be1ea82840c9ee98 2023-02-08 08:55:36,997 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/e0a34fc07b4240ea8d0738115de11e84 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/e0a34fc07b4240ea8d0738115de11e84 2023-02-08 08:55:36,999 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/ee95517ac2154808988acb1615498af6 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf1/ee95517ac2154808988acb1615498af6 2023-02-08 08:55:37,001 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/07d23c8a04e048829fa7296a6962b04e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/07d23c8a04e048829fa7296a6962b04e 2023-02-08 08:55:37,002 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/52bb41d114d048a08c976d1db9299159 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/52bb41d114d048a08c976d1db9299159 2023-02-08 08:55:37,003 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/6e0357a15fa54444b2850ec4d4cf59b7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/6e0357a15fa54444b2850ec4d4cf59b7 2023-02-08 08:55:37,004 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/77d0c3ea6c384fd58d91572f0517e729 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/77d0c3ea6c384fd58d91572f0517e729 2023-02-08 08:55:37,005 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c04bddc539274e34909bd1d433f7b93e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c04bddc539274e34909bd1d433f7b93e 2023-02-08 08:55:37,007 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c6fc41833dce47f6aa1874fee2e4a09d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/c6fc41833dce47f6aa1874fee2e4a09d 2023-02-08 08:55:37,008 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/cf2/cf6c68f5eace4d45880072eb5a9b0945 2023-02-08 08:55:37,011 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae/recovered.edits/85.seqid 2023-02-08 08:55:37,012 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testInvalidSplitKey/44cfa83816aa4855c5a36094c2b511ae 2023-02-08 08:55:37,012 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 08:55:37,014 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:37,015 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-08 08:55:37,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-08 08:55:37,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:37,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-08 08:55:37,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846537018"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:37,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 08:55:37,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 44cfa83816aa4855c5a36094c2b511ae, NAME => 'testInvalidSplitKey,,1675846535060.44cfa83816aa4855c5a36094c2b511ae.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 08:55:37,020 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-08 08:55:37,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846537020"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:37,021 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-08 08:55:37,023 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:37,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 42 msec 2023-02-08 08:55:37,084 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,085 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 08:55:37,086 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-08 08:55:37,099 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=364 (was 345) - Thread LEAK? -, OpenFileDescriptor=742 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 136), ProcessCount=168 (was 168), AvailableMemoryMB=4037 (was 4042) 2023-02-08 08:55:37,112 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=364, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=136, ProcessCount=168, AvailableMemoryMB=4037 2023-02-08 08:55:37,112 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:37,112 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,112 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,114 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:55:37,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:55:37,116 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:37,117 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:37,118 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:37,119 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,120 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 empty. 2023-02-08 08:55:37,120 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,120 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 08:55:37,130 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:37,131 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 69895911dc76808c1aa44b507f07bee2, NAME => 'testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 69895911dc76808c1aa44b507f07bee2, disabling compactions & flushes 2023-02-08 08:55:37,137 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. after waiting 0 ms 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,137 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,137 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 69895911dc76808c1aa44b507f07bee2: Waiting for close lock at 1675846537137Disabling compacts and flushes for region at 1675846537137Disabling writes for close at 1675846537137Writing region close event to WAL at 1675846537137Closed at 1675846537137 2023-02-08 08:55:37,139 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:37,140 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846537140"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846537140"}]},"ts":"1675846537140"} 2023-02-08 08:55:37,141 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:37,141 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:37,142 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846537141"}]},"ts":"1675846537141"} 2023-02-08 08:55:37,142 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 08:55:37,145 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:55:37,146 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:55:37,146 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:55:37,146 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:55:37,146 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:55:37,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, ASSIGN}] 2023-02-08 08:55:37,147 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, ASSIGN 2023-02-08 08:55:37,148 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:55:37,298 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:55:37,299 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=69895911dc76808c1aa44b507f07bee2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846537299"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846537299"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846537299"}]},"ts":"1675846537299"} 2023-02-08 08:55:37,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:37,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 69895911dc76808c1aa44b507f07bee2, NAME => 'testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:37,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:37,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:37,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,457 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,458 DEBUG [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1 2023-02-08 08:55:37,458 DEBUG [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1 2023-02-08 08:55:37,458 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69895911dc76808c1aa44b507f07bee2 columnFamilyName cf1 2023-02-08 08:55:37,459 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] regionserver.HStore(310): Store=69895911dc76808c1aa44b507f07bee2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:37,459 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,460 DEBUG [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2 2023-02-08 08:55:37,460 DEBUG [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2 2023-02-08 08:55:37,460 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69895911dc76808c1aa44b507f07bee2 columnFamilyName cf2 2023-02-08 08:55:37,461 INFO [StoreOpener-69895911dc76808c1aa44b507f07bee2-1] regionserver.HStore(310): Store=69895911dc76808c1aa44b507f07bee2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:37,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:37,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:37,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:37,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 69895911dc76808c1aa44b507f07bee2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66465187, jitterRate=-0.009591534733772278}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:37,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 69895911dc76808c1aa44b507f07bee2: Running coprocessor pre-open hook at 1675846537456Writing region info on filesystem at 1675846537456Initializing all the Stores at 1675846537457 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846537457Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846537457Cleaning up temporary data from old regions at 1675846537462 (+5 ms)Cleaning up detritus from prior splits at 1675846537462Running coprocessor post-open hooks at 1675846537466 (+4 ms)Region opened successfully at 1675846537466 2023-02-08 08:55:37,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2., pid=111, masterSystemTime=1675846537453 2023-02-08 08:55:37,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,468 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=69895911dc76808c1aa44b507f07bee2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,469 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846537468"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846537468"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846537468"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846537468"}]},"ts":"1675846537468"} 2023-02-08 08:55:37,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-08 08:55:37,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 in 169 msec 2023-02-08 08:55:37,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-08 08:55:37,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, ASSIGN in 325 msec 2023-02-08 08:55:37,473 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:37,473 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846537473"}]},"ts":"1675846537473"} 2023-02-08 08:55:37,474 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 08:55:37,476 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:37,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 361 msec 2023-02-08 08:55:37,623 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,623 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:37,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:37,625 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,625 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,625 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:37,625 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:37,625 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,625 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,628 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,628 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,628 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,629 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:37,630 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,630 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:37,632 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:55:37,632 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,633 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,633 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,633 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,633 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,641 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/edb00134414c4364a2cfc641319a30de 2023-02-08 08:55:37,651 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/b89f8d57829c4a1cbaef209c3840d760 2023-02-08 08:55:37,655 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/edb00134414c4364a2cfc641319a30de as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de 2023-02-08 08:55:37,659 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:37,659 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/b89f8d57829c4a1cbaef209c3840d760 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 2023-02-08 08:55:37,663 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:37,663 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 31ms, sequenceid=6, compaction requested=false 2023-02-08 08:55:37,664 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,664 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:37,664 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,664 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:37,664 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:37,664 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,664 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,664 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,666 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,666 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,666 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,666 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76027f05[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:37,668 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:37,668 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,669 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,669 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:37,672 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,672 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,672 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,672 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,672 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:37,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:37,675 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:37,675 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:37,675 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:37,675 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,675 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,677 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,677 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,678 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,678 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,680 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,680 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,680 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,682 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,682 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:37,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,683 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,783 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:37,783 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,783 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:37,794 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,795 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:37,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:37,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:37,797 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,797 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,797 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:37,797 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:37,797 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,797 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,799 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,799 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,800 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,800 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,801 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,802 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,803 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,803 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,803 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,803 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,804 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,804 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,804 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:37,804 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:37,804 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,804 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:37,804 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:37,812 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/2f7a81e7bfb3418ebc05cf61baed417a 2023-02-08 08:55:37,825 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/deb45654d0a34d7a84ee474eaa65ea97 2023-02-08 08:55:37,829 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/2f7a81e7bfb3418ebc05cf61baed417a as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a 2023-02-08 08:55:37,832 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:37,833 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/deb45654d0a34d7a84ee474eaa65ea97 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 2023-02-08 08:55:37,836 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:37,836 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 32ms, sequenceid=19, compaction requested=false 2023-02-08 08:55:37,836 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:37,837 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,837 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:37,837 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:37,837 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,837 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,837 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,841 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,841 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,842 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,842 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21143d18[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:37,843 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:37,843 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,845 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:37,845 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:37,845 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,848 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:37,848 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,848 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,848 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:37,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:37,851 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:37,851 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:37,851 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:37,851 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,851 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,853 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:37,853 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,854 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:37,854 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:37,855 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:37,856 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,856 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,858 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:37,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,858 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:37,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,859 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:37,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:37,959 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:37,960 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,960 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:37,985 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,985 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:37,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:37,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:37,988 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:37,988 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,988 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:37,988 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:37,988 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,988 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,989 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,989 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,989 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,989 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,990 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,990 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,991 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:37,992 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:37,992 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:37,992 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:37,992 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:37,999 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,001 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/2aade8b9a8a844999fc8b792ece4ceb1 2023-02-08 08:55:38,011 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/5599c63fe769479aa4bf8257097682bb 2023-02-08 08:55:38,014 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/2aade8b9a8a844999fc8b792ece4ceb1 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 2023-02-08 08:55:38,018 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:38,018 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/5599c63fe769479aa4bf8257097682bb as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb 2023-02-08 08:55:38,022 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:38,022 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 30ms, sequenceid=32, compaction requested=true 2023-02-08 08:55:38,023 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:38,023 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,024 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,024 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,025 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,025 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:38,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5576868[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:38,026 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:38,026 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:38,028 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,028 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:38,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:38,031 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,032 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,032 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,032 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:38,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:38,034 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:38,035 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:38,035 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:38,035 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,035 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,036 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,036 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,037 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,037 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,039 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,045 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,045 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:38,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:38,046 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,047 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,047 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,147 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:38,147 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,147 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:38,158 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,158 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:38,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:38,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:38,160 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:38,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:38,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,160 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,161 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,161 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,161 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,161 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,162 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,162 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,163 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:38,164 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:38,164 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,164 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:38,164 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:38,165 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,165 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,165 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,165 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,165 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,166 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,174 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/f66164ae58254fda90c937497a874c3e 2023-02-08 08:55:38,184 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/a410b70de65944979ae0bef6a880cec8 2023-02-08 08:55:38,188 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/f66164ae58254fda90c937497a874c3e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e 2023-02-08 08:55:38,191 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:38,191 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/a410b70de65944979ae0bef6a880cec8 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 2023-02-08 08:55:38,194 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:38,195 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 31ms, sequenceid=45, compaction requested=true 2023-02-08 08:55:38,195 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:38,195 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,197 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,197 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,197 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,197 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:38,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@258b062a[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:38,199 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:38,199 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,201 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,201 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:38,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:38,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:38,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:38,202 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,203 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,203 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:38,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:38,206 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:38,206 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:38,206 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:38,206 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,206 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,207 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,208 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,209 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,210 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,210 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:38,214 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,215 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,216 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,216 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,216 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,216 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,315 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:38,315 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,316 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:38,326 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,327 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:38,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:38,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:38,329 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:38,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:38,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,330 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,331 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,331 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,331 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,332 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,332 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:38,332 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,333 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,333 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,333 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:38,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,334 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,335 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,335 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,335 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,335 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,341 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/a5ce145d7fc8412485ba8f79be36e063 2023-02-08 08:55:38,351 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/3d1e01e53c8143da80795ff9d1bf6452 2023-02-08 08:55:38,355 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/a5ce145d7fc8412485ba8f79be36e063 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 2023-02-08 08:55:38,358 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:38,358 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/3d1e01e53c8143da80795ff9d1bf6452 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 2023-02-08 08:55:38,361 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:38,362 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 29ms, sequenceid=58, compaction requested=true 2023-02-08 08:55:38,362 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:38,362 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,362 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:38,362 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:38,362 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,362 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,362 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,373 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,373 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,374 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,374 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,376 ERROR [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f) service to localhost/127.0.0.1:42159 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:55:38,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:38,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4014afc9[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:38,377 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:38,377 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,378 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:38,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:38,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,378 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,379 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,379 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:38,380 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,380 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,381 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:38,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:38,383 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:38,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:38,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:38,383 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,383 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,391 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,392 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,392 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,394 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,394 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,395 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,398 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,398 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:38,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,399 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,499 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:38,499 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,499 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:38,510 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,510 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:38,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:38,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 08:55:38,512 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,512 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,512 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 08:55:38,512 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 08:55:38,513 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,513 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,514 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,514 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,514 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,514 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:38,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:38,516 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:38,516 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,516 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. started... 2023-02-08 08:55:38,516 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:38,517 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,518 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,521 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,521 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,521 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,525 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 2023-02-08 08:55:38,542 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/790f8769e27d49e59edfae308a0f7e0c 2023-02-08 08:55:38,546 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 2023-02-08 08:55:38,549 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:38,550 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/790f8769e27d49e59edfae308a0f7e0c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c 2023-02-08 08:55:38,553 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:38,554 INFO [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 38ms, sequenceid=71, compaction requested=true 2023-02-08 08:55:38,554 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 69895911dc76808c1aa44b507f07bee2: 2023-02-08 08:55:38,554 DEBUG [rs(jenkins-hbase4.apache.org,39055,1675846441504)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 08:55:38,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 08:55:38,556 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,556 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,556 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,556 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 08:55:38,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40e8a62e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:38,558 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 08:55:38,558 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,559 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:38,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:38,559 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 08:55:38,560 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 08:55:38,560 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:38,561 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,561 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,561 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,566 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,566 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,39055,1675846441504': 2023-02-08 08:55:38,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 08:55:38,569 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 08:55:38,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:38,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:38,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 08:55:38,569 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:38,571 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,571 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:38,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,572 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:38,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,573 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,574 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:38,574 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 08:55:38,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 08:55:38,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:38,579 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,580 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:38,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:38,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:38,580 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,581 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 08:55:38,681 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:38,681 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:38,681 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:38,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:38,939 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 08:55:38,940 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:38,941 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:38,942 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:38,942 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:38,942 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:38,942 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:38,942 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:38,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:38,942 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:39,191 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,191 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:39,191 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:39,191 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:39,191 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-08 08:55:39,191 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:39,192 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:39,192 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:39,192 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:39,192 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:39,194 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 08:55:39,194 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:39,194 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:39,195 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:39,195 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:39,195 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,195 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:39,195 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,196 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,199 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,199 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,199 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,199 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,200 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:39,201 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,201 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:39,201 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,201 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,201 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,201 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-08 08:55:39,202 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:39,202 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:39,203 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846569203 2023-02-08 08:55:39,203 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:39,204 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,207 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,208 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,208 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:39,208 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846599208 2023-02-08 08:55:39,208 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846659208 2023-02-08 08:55:39,208 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:55:39,208 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:39,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:39,211 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:39,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:39,458 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,458 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:39,459 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:39,459 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:39,460 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:39,461 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 08:55:39,461 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,462 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:39,463 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:39,464 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,464 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,464 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,464 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,465 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,466 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,467 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:39,467 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,467 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:39,467 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,467 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,467 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,467 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-08 08:55:39,468 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:39,468 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:39,468 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846569468 2023-02-08 08:55:39,468 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:39,470 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,472 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,472 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,472 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:39,473 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846599473 2023-02-08 08:55:39,473 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846659473 2023-02-08 08:55:39,473 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:55:39,473 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:39,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN}] 2023-02-08 08:55:39,475 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:39,475 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:39,475 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:39,723 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:39,723 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:39,724 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:39,724 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:39,725 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 08:55:39,726 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:39,726 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:39,726 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:39,726 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:39,726 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,727 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,728 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,729 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,730 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,731 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:39,731 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,731 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:39,731 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,731 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,731 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,731 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:55:39,732 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:39,732 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:39,733 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846569733 2023-02-08 08:55:39,733 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:39,734 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,738 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,738 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:39,738 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:39,738 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846599738 2023-02-08 08:55:39,738 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846659738 2023-02-08 08:55:39,738 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:55:39,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN}] 2023-02-08 08:55:39,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:39,743 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:39,743 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=69895911dc76808c1aa44b507f07bee2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:39,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846539743"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846539743"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846539743"}]},"ts":"1675846539743"} 2023-02-08 08:55:39,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:39,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:39,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 69895911dc76808c1aa44b507f07bee2, disabling compactions & flushes 2023-02-08 08:55:39,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:39,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:39,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. after waiting 0 ms 2023-02-08 08:55:39,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:39,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 69895911dc76808c1aa44b507f07bee2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:55:39,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/ebf2c6e388094a6fa5367341baec9a83 2023-02-08 08:55:39,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/8721f1a8e7e54b62be93deac51faa659 2023-02-08 08:55:39,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf1/ebf2c6e388094a6fa5367341baec9a83 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 2023-02-08 08:55:39,922 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:39,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/.tmp/cf2/8721f1a8e7e54b62be93deac51faa659 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 2023-02-08 08:55:39,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:39,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 30ms, sequenceid=82, compaction requested=true 2023-02-08 08:55:39,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:55:39,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:39,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. 2023-02-08 08:55:39,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 69895911dc76808c1aa44b507f07bee2: Waiting for close lock at 1675846539897Running coprocessor pre-close hooks at 1675846539897Disabling compacts and flushes for region at 1675846539897Disabling writes for close at 1675846539897Obtaining lock to block concurrent updates at 1675846539897Preparing flush snapshotting stores in 69895911dc76808c1aa44b507f07bee2 at 1675846539897Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846539898 (+1 ms)Flushing stores of testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. at 1675846539898Flushing 69895911dc76808c1aa44b507f07bee2/cf1: creating writer at 1675846539899 (+1 ms)Flushing 69895911dc76808c1aa44b507f07bee2/cf1: appending metadata at 1675846539901 (+2 ms)Flushing 69895911dc76808c1aa44b507f07bee2/cf1: closing flushed file at 1675846539901Flushing 69895911dc76808c1aa44b507f07bee2/cf2: creating writer at 1675846539910 (+9 ms)Flushing 69895911dc76808c1aa44b507f07bee2/cf2: appending metadata at 1675846539911 (+1 ms)Flushing 69895911dc76808c1aa44b507f07bee2/cf2: closing flushed file at 1675846539911Flushing 69895911dc76808c1aa44b507f07bee2/cf1: reopening flushed file at 1675846539920 (+9 ms)Flushing 69895911dc76808c1aa44b507f07bee2/cf2: reopening flushed file at 1675846539923 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 69895911dc76808c1aa44b507f07bee2 in 30ms, sequenceid=82, compaction requested=true at 1675846539927 (+4 ms)Writing region close event to WAL at 1675846539927Running coprocessor post-close hooks at 1675846539930 (+3 ms)Closed at 1675846539931 (+1 ms) 2023-02-08 08:55:39,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:39,933 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED 2023-02-08 08:55:39,933 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846539933"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846539933"}]},"ts":"1675846539933"} 2023-02-08 08:55:39,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-08 08:55:39,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 in 189 msec 2023-02-08 08:55:39,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-08 08:55:39,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN in 198 msec 2023-02-08 08:55:39,939 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:39,939 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:39,939 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:39,988 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:39,989 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:39,989 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:39,990 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:39,991 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 08:55:39,991 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:39,991 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:39,991 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:39,992 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,993 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:39,994 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:39,994 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,994 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:39,994 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,995 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:39,995 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,995 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:39,995 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,995 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:39,996 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,996 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:39,996 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,996 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,997 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:39,997 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,997 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:39,997 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,998 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,998 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:39,998 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:39,998 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,998 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:39,999 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:39,999 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:39,999 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,999 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,999 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:39,999 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:39,999 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:39,999 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 08:55:40,000 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:40,000 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:40,000 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846570000 2023-02-08 08:55:40,001 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:40,002 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,004 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:40,004 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:40,004 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:40,004 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846600004 2023-02-08 08:55:40,004 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846660004 2023-02-08 08:55:40,005 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:40,007 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:40,015 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=69895911dc76808c1aa44b507f07bee2, threads=14 2023-02-08 08:55:40,015 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,015 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,015 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,016 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,017 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,017 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,017 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,017 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,100 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,122 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,122 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,125 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,131 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,168 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,189 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,190 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,191 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,192 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,192 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,193 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,321 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:40,467 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,467 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,467 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 69895911dc76808c1aa44b507f07bee2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:55:40,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:40,517 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:40,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:40,755 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:40,755 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:40,756 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:40,756 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:40,757 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 08:55:40,757 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:40,757 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:40,757 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:40,758 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:40,759 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:40,760 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:40,761 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:40,762 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:40,762 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,763 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,763 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,763 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:40,763 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:40,763 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:55:40,763 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:40,764 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:40,764 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846570764 2023-02-08 08:55:40,764 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:40,766 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,768 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:40,768 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:40,768 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:40,768 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846600768 2023-02-08 08:55:40,768 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846660768 2023-02-08 08:55:40,768 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:40,776 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=69895911dc76808c1aa44b507f07bee2, threads=14 2023-02-08 08:55:40,776 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,776 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,776 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,781 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,781 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,784 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,784 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,784 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,784 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,784 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,785 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,788 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,788 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,788 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,876 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,900 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,914 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,924 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,925 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,925 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,931 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,931 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,934 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,934 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,934 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,935 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,936 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,936 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c for region: 69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:40,936 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 69895911dc76808c1aa44b507f07bee2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:55:40,964 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:40,968 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:40,969 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:40,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:40,969 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:40,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:40,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:41,018 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,018 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:41,018 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:41,019 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:41,019 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:41,019 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:41,019 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:41,019 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:41,019 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:41,020 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:41,021 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:41,022 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,023 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,024 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:41,025 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,026 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:41,026 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:41,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:41,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,026 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,026 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,026 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:55:41,027 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:41,027 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:41,027 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846571027 2023-02-08 08:55:41,027 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:41,029 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,031 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,031 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,031 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:41,031 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846601031 2023-02-08 08:55:41,031 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846661031 2023-02-08 08:55:41,031 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:41,034 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:41,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:41,036 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:41,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:41,281 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:41,282 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:41,282 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:41,283 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:41,284 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 08:55:41,284 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:41,284 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:41,284 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:41,284 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,285 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,286 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,287 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,288 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:41,289 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,289 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,289 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,289 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,289 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:55:41,290 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:41,290 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:41,291 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846571291 2023-02-08 08:55:41,292 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:41,292 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,294 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,294 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,294 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:41,294 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846601294 2023-02-08 08:55:41,295 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846661295 2023-02-08 08:55:41,295 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:41,295 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:41,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846541296"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675846541296"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675846541296"}]},"ts":"1675846541296"} 2023-02-08 08:55:41,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846541296"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846541296"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846541296"}]},"ts":"1675846541296"} 2023-02-08 08:55:41,297 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846541296"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846541296"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846541296"}]},"ts":"1675846541296"} 2023-02-08 08:55:41,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:41,298 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:41,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:41,373 ERROR [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 39265e67-bcfa-4780-a199-0487aaa87ffd) service to localhost/127.0.0.1:42159 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:55:41,545 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,545 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:41,545 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:41,545 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:41,545 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:41,546 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:41,546 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:41,546 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:41,546 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:41,546 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:41,548 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 08:55:41,548 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:41,548 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:41,549 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:41,549 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:41,549 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:41,549 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:41,549 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,550 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:41,550 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,550 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,550 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:41,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,551 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:41,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:41,552 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:41,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,553 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,554 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,554 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:41,554 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,554 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,555 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,555 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,555 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:41,556 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,556 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,556 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,556 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,556 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,557 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:41,557 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,557 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:41,557 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:41,557 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:41,558 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,558 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,558 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,558 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,558 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,558 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-08 08:55:41,559 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:41,559 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:41,562 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846571562 2023-02-08 08:55:41,564 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,564 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:41,566 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,566 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,566 INFO [Listener at localhost/33105] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504, table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:41,567 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=986dd857f3e95aa74dd0703e5f89b329, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:41,567 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=b821f0ad2df6d083af1da61bc5f35ed1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:41,567 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:41,567 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846601567 2023-02-08 08:55:41,567 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846661567 2023-02-08 08:55:41,567 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:41,568 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846541567"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675846541567"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675846541567"}]},"ts":"1675846541567"} 2023-02-08 08:55:41,568 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846541567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846541567"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846541567"}]},"ts":"1675846541567"} 2023-02-08 08:55:41,568 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846541567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846541567"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846541567"}]},"ts":"1675846541567"} 2023-02-08 08:55:41,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:41,571 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:41,571 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:41,572 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:41,817 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,817 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:41,818 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:41,818 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:41,819 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:41,820 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-08 08:55:41,820 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:41,820 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:41,820 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:41,820 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:41,821 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:41,822 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:41,823 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:41,824 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:41,825 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,825 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,825 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,825 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:41,825 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 08:55:41,826 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:41,826 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:41,827 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846571827 2023-02-08 08:55:41,827 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:41,828 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,830 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,830 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:41,831 INFO [Listener at localhost/33105] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504, table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:41,831 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=986dd857f3e95aa74dd0703e5f89b329, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:41,831 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=b821f0ad2df6d083af1da61bc5f35ed1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:41,831 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:41,831 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846601831 2023-02-08 08:55:41,831 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846661831 2023-02-08 08:55:41,831 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 08:55:41,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:41,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN}] 2023-02-08 08:55:41,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:41,833 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:41,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:41,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:42,081 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,081 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:42,081 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:42,082 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:42,082 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:42,082 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:42,082 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:42,082 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:42,082 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:42,083 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:42,084 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:42,085 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,086 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:42,087 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,087 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,087 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,087 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:42,087 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,088 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,088 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,088 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,088 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:42,089 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,089 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,089 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,089 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,089 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,090 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:42,090 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,090 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:42,090 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,090 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:42,090 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,091 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,091 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,091 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,091 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,091 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-08 08:55:42,092 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:42,092 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:42,092 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846572092 2023-02-08 08:55:42,092 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:42,094 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,096 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:42,097 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:42,097 INFO [Listener at localhost/33105] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504, table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:42,097 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=986dd857f3e95aa74dd0703e5f89b329, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:42,097 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=b821f0ad2df6d083af1da61bc5f35ed1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 08:55:42,097 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:42,097 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846602097 2023-02-08 08:55:42,097 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846662097 2023-02-08 08:55:42,097 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 08:55:42,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN}] 2023-02-08 08:55:42,098 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:42,099 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN 2023-02-08 08:55:42,100 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:55:42,100 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN 2023-02-08 08:55:42,101 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504; forceNewPlan=false, retain=false 2023-02-08 08:55:42,250 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:55:42,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=986dd857f3e95aa74dd0703e5f89b329, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,252 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846542252"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846542252"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846542252"}]},"ts":"1675846542252"} 2023-02-08 08:55:42,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:42,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b821f0ad2df6d083af1da61bc5f35ed1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,255 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846542255"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846542255"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846542255"}]},"ts":"1675846542255"} 2023-02-08 08:55:42,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:42,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b821f0ad2df6d083af1da61bc5f35ed1, NAME => 'testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 08:55:42,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:42,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:42,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,408 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,409 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1 2023-02-08 08:55:42,409 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1 2023-02-08 08:55:42,410 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b821f0ad2df6d083af1da61bc5f35ed1 columnFamilyName cf1 2023-02-08 08:55:42,420 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-top 2023-02-08 08:55:42,423 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-top 2023-02-08 08:55:42,426 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-top 2023-02-08 08:55:42,428 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-top 2023-02-08 08:55:42,431 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-top 2023-02-08 08:55:42,434 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-top 2023-02-08 08:55:42,436 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-top 2023-02-08 08:55:42,437 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(310): Store=b821f0ad2df6d083af1da61bc5f35ed1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:42,437 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,437 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2 2023-02-08 08:55:42,437 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2 2023-02-08 08:55:42,438 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b821f0ad2df6d083af1da61bc5f35ed1 columnFamilyName cf2 2023-02-08 08:55:42,448 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-top 2023-02-08 08:55:42,451 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-top 2023-02-08 08:55:42,453 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-top 2023-02-08 08:55:42,457 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-top 2023-02-08 08:55:42,460 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-top 2023-02-08 08:55:42,467 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-top 2023-02-08 08:55:42,470 DEBUG [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-top 2023-02-08 08:55:42,470 INFO [StoreOpener-b821f0ad2df6d083af1da61bc5f35ed1-1] regionserver.HStore(310): Store=b821f0ad2df6d083af1da61bc5f35ed1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:42,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:42,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b821f0ad2df6d083af1da61bc5f35ed1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74084052, jitterRate=0.10393840074539185}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:42,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b821f0ad2df6d083af1da61bc5f35ed1: Running coprocessor pre-open hook at 1675846542408Writing region info on filesystem at 1675846542408Initializing all the Stores at 1675846542408Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846542408Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846542408Cleaning up temporary data from old regions at 1675846542472 (+64 ms)Cleaning up detritus from prior splits at 1675846542473 (+1 ms)Running coprocessor post-open hooks at 1675846542475 (+2 ms)Region opened successfully at 1675846542475 2023-02-08 08:55:42,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1., pid=118, masterSystemTime=1675846542404 2023-02-08 08:55:42,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:42,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:42,477 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:42,477 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:42,477 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,477 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1912): b821f0ad2df6d083af1da61bc5f35ed1/cf1 is initiating minor compaction (all files) 2023-02-08 08:55:42,477 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,477 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b821f0ad2df6d083af1da61bc5f35ed1/cf1 in testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,477 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1912): b821f0ad2df6d083af1da61bc5f35ed1/cf2 is initiating minor compaction (all files) 2023-02-08 08:55:42,477 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b821f0ad2df6d083af1da61bc5f35ed1/cf2 in testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,477 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/.tmp, totalSize=34.9 K 2023-02-08 08:55:42,478 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/.tmp, totalSize=34.9 K 2023-02-08 08:55:42,478 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846537619 2023-02-08 08:55:42,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:42,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,478 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846537619 2023-02-08 08:55:42,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 986dd857f3e95aa74dd0703e5f89b329, NAME => 'testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 08:55:42,479 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting 2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846537785 2023-02-08 08:55:42,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:42,479 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846537785 2023-02-08 08:55:42,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,479 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting 2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846537970 2023-02-08 08:55:42,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:42,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,479 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846537970 2023-02-08 08:55:42,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b821f0ad2df6d083af1da61bc5f35ed1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846542479"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846542479"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846542479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846542479"}]},"ts":"1675846542479"} 2023-02-08 08:55:42,480 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846538149 2023-02-08 08:55:42,480 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846538149 2023-02-08 08:55:42,480 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,480 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846538317 2023-02-08 08:55:42,481 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846538317 2023-02-08 08:55:42,481 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846538501 2023-02-08 08:55:42,481 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting 9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846538501 2023-02-08 08:55:42,481 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1 2023-02-08 08:55:42,481 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1 2023-02-08 08:55:42,481 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846538682 2023-02-08 08:55:42,482 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] compactions.Compactor(207): Compacting ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846538682 2023-02-08 08:55:42,482 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 986dd857f3e95aa74dd0703e5f89b329 columnFamilyName cf1 2023-02-08 08:55:42,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-08 08:55:42,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504 in 225 msec 2023-02-08 08:55:42,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN in 385 msec 2023-02-08 08:55:42,493 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] throttle.PressureAwareThroughputController(145): b821f0ad2df6d083af1da61bc5f35ed1#cf2#compaction#106 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:42,495 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] throttle.PressureAwareThroughputController(145): b821f0ad2df6d083af1da61bc5f35ed1#cf1#compaction#107 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:42,514 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-bottom 2023-02-08 08:55:42,517 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/.tmp/cf2/64734ba8b93f47579d0d7877a165bb63 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/64734ba8b93f47579d0d7877a165bb63 2023-02-08 08:55:42,517 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-bottom 2023-02-08 08:55:42,520 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-bottom 2023-02-08 08:55:42,521 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b821f0ad2df6d083af1da61bc5f35ed1/cf2 of b821f0ad2df6d083af1da61bc5f35ed1 into 64734ba8b93f47579d0d7877a165bb63(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:42,521 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b821f0ad2df6d083af1da61bc5f35ed1: 2023-02-08 08:55:42,521 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1., storeName=b821f0ad2df6d083af1da61bc5f35ed1/cf2, priority=9, startTime=1675846542477; duration=0sec 2023-02-08 08:55:42,521 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:42,523 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-bottom 2023-02-08 08:55:42,527 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-bottom 2023-02-08 08:55:42,529 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-bottom 2023-02-08 08:55:42,531 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-bottom 2023-02-08 08:55:42,532 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(310): Store=986dd857f3e95aa74dd0703e5f89b329/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:42,532 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,532 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2 2023-02-08 08:55:42,533 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2 2023-02-08 08:55:42,533 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 986dd857f3e95aa74dd0703e5f89b329 columnFamilyName cf2 2023-02-08 08:55:42,544 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-bottom 2023-02-08 08:55:42,546 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-bottom 2023-02-08 08:55:42,549 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-bottom 2023-02-08 08:55:42,552 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-bottom 2023-02-08 08:55:42,555 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-bottom 2023-02-08 08:55:42,557 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-bottom 2023-02-08 08:55:42,560 DEBUG [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-bottom 2023-02-08 08:55:42,560 INFO [StoreOpener-986dd857f3e95aa74dd0703e5f89b329-1] regionserver.HStore(310): Store=986dd857f3e95aa74dd0703e5f89b329/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:42,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:42,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:42,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 986dd857f3e95aa74dd0703e5f89b329; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68109292, jitterRate=0.014907538890838623}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:42,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 986dd857f3e95aa74dd0703e5f89b329: Running coprocessor pre-open hook at 1675846542479Writing region info on filesystem at 1675846542479Initializing all the Stores at 1675846542480 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846542480Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846542480Cleaning up temporary data from old regions at 1675846542562 (+82 ms)Cleaning up detritus from prior splits at 1675846542562Running coprocessor post-open hooks at 1675846542564 (+2 ms)Region opened successfully at 1675846542564 2023-02-08 08:55:42,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329., pid=117, masterSystemTime=1675846542404 2023-02-08 08:55:42,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:42,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:42,566 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:42,566 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,566 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1912): 986dd857f3e95aa74dd0703e5f89b329/cf1 is initiating minor compaction (all files) 2023-02-08 08:55:42,566 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 986dd857f3e95aa74dd0703e5f89b329/cf1 in testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,567 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/.tmp, totalSize=34.9 K 2023-02-08 08:55:42,567 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846537619 2023-02-08 08:55:42,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,568 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846537785 2023-02-08 08:55:42,568 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846537970 2023-02-08 08:55:42,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=986dd857f3e95aa74dd0703e5f89b329, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846542568"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846542568"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846542568"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846542568"}]},"ts":"1675846542568"} 2023-02-08 08:55:42,569 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846538149 2023-02-08 08:55:42,569 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846538317 2023-02-08 08:55:42,570 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846538501 2023-02-08 08:55:42,570 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846538682 2023-02-08 08:55:42,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-08 08:55:42,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504 in 317 msec 2023-02-08 08:55:42,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-08 08:55:42,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN in 473 msec 2023-02-08 08:55:42,573 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 08:55:42,573 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:42,573 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 08:55:42,579 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] throttle.PressureAwareThroughputController(145): 986dd857f3e95aa74dd0703e5f89b329#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:42,593 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/.tmp/cf1/b2a3369fb940497abb27d369f611b931 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/b2a3369fb940497abb27d369f611b931 2023-02-08 08:55:42,597 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 986dd857f3e95aa74dd0703e5f89b329/cf1 of 986dd857f3e95aa74dd0703e5f89b329 into b2a3369fb940497abb27d369f611b931(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:42,597 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 986dd857f3e95aa74dd0703e5f89b329: 2023-02-08 08:55:42,597 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329., storeName=986dd857f3e95aa74dd0703e5f89b329/cf1, priority=9, startTime=1675846542565; duration=0sec 2023-02-08 08:55:42,597 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:42,597 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:42,597 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,597 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1912): 986dd857f3e95aa74dd0703e5f89b329/cf2 is initiating minor compaction (all files) 2023-02-08 08:55:42,597 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 986dd857f3e95aa74dd0703e5f89b329/cf2 in testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:42,598 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/.tmp, totalSize=34.9 K 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:42,598 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846537619 2023-02-08 08:55:42,598 ERROR [Listener at localhost/33105] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 08:55:42,598 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 08:55:42,598 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846537785 2023-02-08 08:55:42,599 INFO [Listener at localhost/33105] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:55:42,599 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846537970 2023-02-08 08:55:42,599 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 08:55:42,599 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846538149 2023-02-08 08:55:42,600 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846538317 2023-02-08 08:55:42,600 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846538501 2023-02-08 08:55:42,600 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-08 08:55:42,600 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 08:55:42,600 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] compactions.Compactor(207): Compacting 8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846538682 2023-02-08 08:55:42,600 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 08:55:42,601 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=366926f8e963611bc9739ee35ac1108b, daughterA=67cb0b0aee3415fbb18966a88c15b8b5, daughterB=0c013ea68aeb4bc9d7e5174c5e923659 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 08:55:42,602 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,603 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4c3aeb6b729e6681efb5e73977d7b2cf, daughterA=4640d7a44b2f6cb5eae699f90c14ecc9, daughterB=df6b1a9175455ee27dbcef5fc4d3c6b6 2023-02-08 08:55:42,603 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,603 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 08:55:42,603 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,603 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=14b4c549942f07985130891b41879838, daughterA=7c2d1fee6eaa17c701bd1dfe9d595207, daughterB=b319ee264d2fa548e3cb0e2cf2a76764 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=14b4c549942f07985130891b41879838, ASSIGN 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=63d7e5dfe0dba860d1578c977cb72017, daughterA=81a836dbf579d2f8cf4ba6c294cf81ed, daughterB=01f3ff9798c228c40b5ec93eee2dc700 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,604 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=2c6d5653f20710b77eab401d7e0f610d, daughterA=b5a2b4f4020c2cbcbab2bc91399ebe33, daughterB=d8a3a28ddb9422c08833bf51afe408bf 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,605 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,44651,1675846441425, table=testRollbackForSplitTableRegionWithReplica, region=786f8bf894e79e4fdc03875423343e48 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=42130720bfc979feffebd5919715188a, daughterA=29976ed74b3a2f38081b224c7560c2f0, daughterB=3f7e74669aaf2a94184c596bd70bd2ca 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,606 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 08:55:42,607 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,608 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 08:55:42,609 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,609 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,609 INFO [Listener at localhost/33105] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 69895911dc76808c1aa44b507f07bee2, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 DEBUG [Listener at localhost/33105] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 08:55:42,609 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-08 08:55:42,610 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] throttle.PressureAwareThroughputController(145): 986dd857f3e95aa74dd0703e5f89b329#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:42,611 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 08:55:42,611 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,36805,1675846441472, table=hbase:meta, region=1588230740 2023-02-08 08:55:42,612 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846572612 2023-02-08 08:55:42,613 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:42,613 DEBUG [Listener at localhost/33105] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:55:42,616 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=9a7e5b1dad112219d73dc56e1f1e8ebf, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:42,616 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=69895911dc76808c1aa44b507f07bee2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=2 2023-02-08 08:55:42,616 INFO [Listener at localhost/33105] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,39055,1675846441504, table=testRecoveryAndDoubleExecution, region=69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:42,616 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=986dd857f3e95aa74dd0703e5f89b329, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=86 2023-02-08 08:55:42,616 DEBUG [Listener at localhost/33105] assignment.RegionStateStore(147): Load hbase:meta entry region=b821f0ad2df6d083af1da61bc5f35ed1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39055,1675846441504, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504, openSeqNum=86 2023-02-08 08:55:42,616 INFO [Listener at localhost/33105] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 08:55:42,616 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846602616 2023-02-08 08:55:42,616 INFO [Listener at localhost/33105] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846662616 2023-02-08 08:55:42,616 INFO [Listener at localhost/33105] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 08:55:42,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 08:55:42,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=69895911dc76808c1aa44b507f07bee2, daughterA=986dd857f3e95aa74dd0703e5f89b329, daughterB=b821f0ad2df6d083af1da61bc5f35ed1 in 3.6770 sec 2023-02-08 08:55:42,628 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/.tmp/cf2/d807eb6d5bca46a78a815db332203271 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/d807eb6d5bca46a78a815db332203271 2023-02-08 08:55:42,632 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 986dd857f3e95aa74dd0703e5f89b329/cf2 of 986dd857f3e95aa74dd0703e5f89b329 into d807eb6d5bca46a78a815db332203271(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:42,632 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 986dd857f3e95aa74dd0703e5f89b329: 2023-02-08 08:55:42,632 INFO [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329., storeName=986dd857f3e95aa74dd0703e5f89b329/cf2, priority=9, startTime=1675846542565; duration=0sec 2023-02-08 08:55:42,632 DEBUG [RS:2;jenkins-hbase4:39055-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:42,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39055] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:40188 deadline: 1675846602868, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2. is not online on jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:42,919 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/.tmp/cf1/b0821071808c4694bbe6e26d67ca0c73 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/b0821071808c4694bbe6e26d67ca0c73 2023-02-08 08:55:42,923 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b821f0ad2df6d083af1da61bc5f35ed1/cf1 of b821f0ad2df6d083af1da61bc5f35ed1 into b0821071808c4694bbe6e26d67ca0c73(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:42,924 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b821f0ad2df6d083af1da61bc5f35ed1: 2023-02-08 08:55:42,924 INFO [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1., storeName=b821f0ad2df6d083af1da61bc5f35ed1/cf1, priority=9, startTime=1675846542476; duration=0sec 2023-02-08 08:55:42,924 DEBUG [RS:2;jenkins-hbase4:39055-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:42,978 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:55:42,984 INFO [Listener at localhost/33105] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 08:55:42,985 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:55:42,993 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:42,993 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:42,993 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:42,994 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:42,994 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:42,995 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-08 08:55:42,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-08 08:55:42,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:42,997 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:42,997 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:42,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 08:55:42,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846542998"}]},"ts":"1675846542998"} 2023-02-08 08:55:42,999 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 08:55:43,001 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-08 08:55:43,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, UNASSIGN}] 2023-02-08 08:55:43,002 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, UNASSIGN 2023-02-08 08:55:43,003 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b821f0ad2df6d083af1da61bc5f35ed1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,003 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846543003"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846543003"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846543003"}]},"ts":"1675846543003"} 2023-02-08 08:55:43,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:43,005 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, UNASSIGN 2023-02-08 08:55:43,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=986dd857f3e95aa74dd0703e5f89b329, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846543006"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846543006"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846543006"}]},"ts":"1675846543006"} 2023-02-08 08:55:43,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504}] 2023-02-08 08:55:43,098 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,098 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 08:55:43,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:43,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b821f0ad2df6d083af1da61bc5f35ed1, disabling compactions & flushes 2023-02-08 08:55:43,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:43,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:43,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. after waiting 0 ms 2023-02-08 08:55:43,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:43,156 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-top] to archive 2023-02-08 08:55:43,157 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:43,158 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,159 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,160 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,161 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,162 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,163 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,164 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,165 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-top] to archive 2023-02-08 08:55:43,166 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:43,167 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,168 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,169 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,170 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,170 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,171 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,172 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:55:43,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:43,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1. 2023-02-08 08:55:43,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b821f0ad2df6d083af1da61bc5f35ed1: Waiting for close lock at 1675846543155Running coprocessor pre-close hooks at 1675846543155Disabling compacts and flushes for region at 1675846543155Disabling writes for close at 1675846543156 (+1 ms)Writing region close event to WAL at 1675846543173 (+17 ms)Running coprocessor post-close hooks at 1675846543175 (+2 ms)Closed at 1675846543175 2023-02-08 08:55:43,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:43,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:43,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 986dd857f3e95aa74dd0703e5f89b329, disabling compactions & flushes 2023-02-08 08:55:43,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:43,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:43,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. after waiting 0 ms 2023-02-08 08:55:43,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:43,178 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b821f0ad2df6d083af1da61bc5f35ed1, regionState=CLOSED 2023-02-08 08:55:43,178 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846543178"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846543178"}]},"ts":"1675846543178"} 2023-02-08 08:55:43,179 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83-bottom] to archive 2023-02-08 08:55:43,179 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:43,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-08 08:55:43,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure b821f0ad2df6d083af1da61bc5f35ed1, server=jenkins-hbase4.apache.org,39055,1675846441504 in 175 msec 2023-02-08 08:55:43,181 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b821f0ad2df6d083af1da61bc5f35ed1, UNASSIGN in 180 msec 2023-02-08 08:55:43,182 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,183 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,184 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,185 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,185 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,186 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,188 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659-bottom] to archive 2023-02-08 08:55:43,188 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:43,189 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,190 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,191 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,192 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,192 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,193 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,194 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:55:43,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:43,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329. 2023-02-08 08:55:43,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 986dd857f3e95aa74dd0703e5f89b329: Waiting for close lock at 1675846543178Running coprocessor pre-close hooks at 1675846543178Disabling compacts and flushes for region at 1675846543178Disabling writes for close at 1675846543178Writing region close event to WAL at 1675846543195 (+17 ms)Running coprocessor post-close hooks at 1675846543197 (+2 ms)Closed at 1675846543197 2023-02-08 08:55:43,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:43,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=986dd857f3e95aa74dd0703e5f89b329, regionState=CLOSED 2023-02-08 08:55:43,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846543199"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846543199"}]},"ts":"1675846543199"} 2023-02-08 08:55:43,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-08 08:55:43,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 986dd857f3e95aa74dd0703e5f89b329, server=jenkins-hbase4.apache.org,39055,1675846441504 in 194 msec 2023-02-08 08:55:43,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-08 08:55:43,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=986dd857f3e95aa74dd0703e5f89b329, UNASSIGN in 200 msec 2023-02-08 08:55:43,203 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846543203"}]},"ts":"1675846543203"} 2023-02-08 08:55:43,204 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 08:55:43,205 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-08 08:55:43,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 210 msec 2023-02-08 08:55:43,299 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,299 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 08:55:43,300 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-08 08:55:43,300 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,300 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-08 08:55:43,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,303 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,303 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,303 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,304 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 08:55:43,308 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,308 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:43,309 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:43,312 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/recovered.edits] 2023-02-08 08:55:43,313 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/recovered.edits] 2023-02-08 08:55:43,314 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/recovered.edits] 2023-02-08 08:55:43,320 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/.regioninfo to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/.regioninfo 2023-02-08 08:55:43,320 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2aade8b9a8a844999fc8b792ece4ceb1 2023-02-08 08:55:43,320 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/.regioninfo to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/.regioninfo 2023-02-08 08:55:43,321 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/2f7a81e7bfb3418ebc05cf61baed417a 2023-02-08 08:55:43,323 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0 2023-02-08 08:55:43,323 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,323 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2aade8b9a8a844999fc8b792ece4ceb1.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,324 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/a5ce145d7fc8412485ba8f79be36e063 2023-02-08 08:55:43,325 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,325 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/2f7a81e7bfb3418ebc05cf61baed417a.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,326 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/ebf2c6e388094a6fa5367341baec9a83 2023-02-08 08:55:43,327 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,327 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/9dfec09dcc3543c3a9f5cdb63e3deaa0.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,328 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/edb00134414c4364a2cfc641319a30de 2023-02-08 08:55:43,329 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,329 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/a5ce145d7fc8412485ba8f79be36e063.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,330 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf1/f66164ae58254fda90c937497a874c3e 2023-02-08 08:55:43,331 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,331 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/ebf2c6e388094a6fa5367341baec9a83.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,332 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,332 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/edb00134414c4364a2cfc641319a30de.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,333 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/3d1e01e53c8143da80795ff9d1bf6452 2023-02-08 08:55:43,334 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,334 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf1/f66164ae58254fda90c937497a874c3e.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,334 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/5599c63fe769479aa4bf8257097682bb 2023-02-08 08:55:43,336 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/790f8769e27d49e59edfae308a0f7e0c 2023-02-08 08:55:43,337 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,337 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/3d1e01e53c8143da80795ff9d1bf6452.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,337 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/8721f1a8e7e54b62be93deac51faa659 2023-02-08 08:55:43,338 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,339 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/5599c63fe769479aa4bf8257097682bb.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,339 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/a410b70de65944979ae0bef6a880cec8 2023-02-08 08:55:43,340 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,340 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/790f8769e27d49e59edfae308a0f7e0c.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,341 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/b89f8d57829c4a1cbaef209c3840d760 2023-02-08 08:55:43,342 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,342 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/8721f1a8e7e54b62be93deac51faa659.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,342 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/cf2/deb45654d0a34d7a84ee474eaa65ea97 2023-02-08 08:55:43,343 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,343 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/a410b70de65944979ae0bef6a880cec8.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,345 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,345 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/b89f8d57829c4a1cbaef209c3840d760.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,346 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2/recovered.edits/85.seqid 2023-02-08 08:55:43,347 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/b821f0ad2df6d083af1da61bc5f35ed1/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,347 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,347 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/986dd857f3e95aa74dd0703e5f89b329/cf2/deb45654d0a34d7a84ee474eaa65ea97.69895911dc76808c1aa44b507f07bee2 2023-02-08 08:55:43,349 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/b0821071808c4694bbe6e26d67ca0c73 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf1/b0821071808c4694bbe6e26d67ca0c73 2023-02-08 08:55:43,350 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/b2a3369fb940497abb27d369f611b931 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf1/b2a3369fb940497abb27d369f611b931 2023-02-08 08:55:43,352 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/64734ba8b93f47579d0d7877a165bb63 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/cf2/64734ba8b93f47579d0d7877a165bb63 2023-02-08 08:55:43,353 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/d807eb6d5bca46a78a815db332203271 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/cf2/d807eb6d5bca46a78a815db332203271 2023-02-08 08:55:43,355 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1/recovered.edits/90.seqid 2023-02-08 08:55:43,355 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/b821f0ad2df6d083af1da61bc5f35ed1 2023-02-08 08:55:43,356 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329/recovered.edits/90.seqid 2023-02-08 08:55:43,357 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testRecoveryAndDoubleExecution/986dd857f3e95aa74dd0703e5f89b329 2023-02-08 08:55:43,357 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 08:55:43,359 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,361 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-08 08:55:43,362 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-08 08:55:43,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-08 08:55:43,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846543363"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:43,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846543363"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:43,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846543363"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:43,365 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:55:43,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 69895911dc76808c1aa44b507f07bee2, NAME => 'testRecoveryAndDoubleExecution,,1675846537115.69895911dc76808c1aa44b507f07bee2.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 986dd857f3e95aa74dd0703e5f89b329, NAME => 'testRecoveryAndDoubleExecution,,1675846538940.986dd857f3e95aa74dd0703e5f89b329.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b821f0ad2df6d083af1da61bc5f35ed1, NAME => 'testRecoveryAndDoubleExecution,41,1675846538940.b821f0ad2df6d083af1da61bc5f35ed1.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 08:55:43,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-08 08:55:43,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846543365"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:43,366 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-08 08:55:43,368 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 08:55:43,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 67 msec 2023-02-08 08:55:43,377 ERROR [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f) service to localhost/127.0.0.1:42159 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:55:43,404 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,404 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 08:55:43,405 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-08 08:55:43,419 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=371 (was 364) - Thread LEAK? -, OpenFileDescriptor=833 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=133 (was 136), ProcessCount=168 (was 168), AvailableMemoryMB=3978 (was 4037) 2023-02-08 08:55:43,431 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=371, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=133, ProcessCount=168, AvailableMemoryMB=3978 2023-02-08 08:55:43,432 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:43,432 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,432 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,433 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:55:43,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:55:43,435 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:43,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:43,438 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,438 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 empty. 2023-02-08 08:55:43,438 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 08:55:43,446 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:43,447 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1349470b3394f45184feff0caaa8a0f5, NAME => 'testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:55:43,453 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:43,453 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 1349470b3394f45184feff0caaa8a0f5, disabling compactions & flushes 2023-02-08 08:55:43,453 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,453 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,453 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. after waiting 0 ms 2023-02-08 08:55:43,453 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,454 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,454 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 1349470b3394f45184feff0caaa8a0f5: Waiting for close lock at 1675846543453Disabling compacts and flushes for region at 1675846543453Disabling writes for close at 1675846543453Writing region close event to WAL at 1675846543454 (+1 ms)Closed at 1675846543454 2023-02-08 08:55:43,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:43,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846543456"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846543456"}]},"ts":"1675846543456"} 2023-02-08 08:55:43,457 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:43,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:43,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846543457"}]},"ts":"1675846543457"} 2023-02-08 08:55:43,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-08 08:55:43,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:55:43,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:55:43,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:55:43,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:55:43,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:55:43,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, ASSIGN}] 2023-02-08 08:55:43,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, ASSIGN 2023-02-08 08:55:43,464 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:43,614 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:55:43,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=1349470b3394f45184feff0caaa8a0f5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846543616"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846543616"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846543616"}]},"ts":"1675846543616"} 2023-02-08 08:55:43,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 1349470b3394f45184feff0caaa8a0f5, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:43,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1349470b3394f45184feff0caaa8a0f5, NAME => 'testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:43,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:43,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:43,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,775 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,776 DEBUG [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1 2023-02-08 08:55:43,776 DEBUG [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1 2023-02-08 08:55:43,776 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1349470b3394f45184feff0caaa8a0f5 columnFamilyName cf1 2023-02-08 08:55:43,777 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] regionserver.HStore(310): Store=1349470b3394f45184feff0caaa8a0f5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:43,777 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,778 DEBUG [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2 2023-02-08 08:55:43,778 DEBUG [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2 2023-02-08 08:55:43,778 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1349470b3394f45184feff0caaa8a0f5 columnFamilyName cf2 2023-02-08 08:55:43,778 INFO [StoreOpener-1349470b3394f45184feff0caaa8a0f5-1] regionserver.HStore(310): Store=1349470b3394f45184feff0caaa8a0f5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:43,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:43,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:43,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1349470b3394f45184feff0caaa8a0f5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72942320, jitterRate=0.08692526817321777}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:43,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1349470b3394f45184feff0caaa8a0f5: Running coprocessor pre-open hook at 1675846543774Writing region info on filesystem at 1675846543774Initializing all the Stores at 1675846543775 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846543775Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846543775Cleaning up temporary data from old regions at 1675846543780 (+5 ms)Cleaning up detritus from prior splits at 1675846543780Running coprocessor post-open hooks at 1675846543783 (+3 ms)Region opened successfully at 1675846543783 2023-02-08 08:55:43,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5., pid=127, masterSystemTime=1675846543768 2023-02-08 08:55:43,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=1349470b3394f45184feff0caaa8a0f5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,786 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846543785"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846543785"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846543785"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846543785"}]},"ts":"1675846543785"} 2023-02-08 08:55:43,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-08 08:55:43,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 1349470b3394f45184feff0caaa8a0f5, server=jenkins-hbase4.apache.org,44651,1675846441425 in 170 msec 2023-02-08 08:55:43,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-08 08:55:43,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, ASSIGN in 325 msec 2023-02-08 08:55:43,790 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:43,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846543790"}]},"ts":"1675846543790"} 2023-02-08 08:55:43,791 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-08 08:55:43,793 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:43,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 359 msec 2023-02-08 08:55:43,941 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:43,941 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:43,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:43,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,943 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:43,943 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:43,943 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:43,943 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:43,943 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,943 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,944 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,944 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,945 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,944 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,945 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:43,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:43,946 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:43,946 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,947 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:43,947 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:55:43,950 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,951 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,951 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,951 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:43,951 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,951 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:43,956 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/c1124596cd0e4d34903f6115ee5334be 2023-02-08 08:55:43,965 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 2023-02-08 08:55:43,969 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/c1124596cd0e4d34903f6115ee5334be as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be 2023-02-08 08:55:43,971 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:43,972 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 2023-02-08 08:55:43,975 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:43,975 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 28ms, sequenceid=6, compaction requested=false 2023-02-08 08:55:43,975 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:43,976 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:43,976 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,977 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:43,978 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,978 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:43,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e429273[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:43,979 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:43,979 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,980 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,980 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:43,980 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:43,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:43,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:43,983 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,983 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:43,984 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:43,984 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:43,984 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:43,984 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:43,984 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:43,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:43,987 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:43,987 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:43,987 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:43,987 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,987 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:43,988 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,989 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,989 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,989 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,991 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:43,991 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,991 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:43,997 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:43,997 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:43,998 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:43,999 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:43,999 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,099 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:44,099 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,099 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:44,109 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,109 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:44,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:44,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,111 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:44,111 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,111 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:44,111 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:44,112 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,112 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,113 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,113 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,113 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:44,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:44,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:44,115 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:44,115 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:44,115 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:44,116 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,523 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/873b1a56a3a4438283e849d1edfe8b61 2023-02-08 08:55:44,533 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/b8cce518b1274ce8829539280a33af08 2023-02-08 08:55:44,537 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/873b1a56a3a4438283e849d1edfe8b61 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61 2023-02-08 08:55:44,541 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:44,542 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/b8cce518b1274ce8829539280a33af08 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08 2023-02-08 08:55:44,546 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:44,547 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 432ms, sequenceid=19, compaction requested=false 2023-02-08 08:55:44,547 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:44,547 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:44,547 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:44,547 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:44,547 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,547 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:44,547 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:44,549 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,549 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:44,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38561c7[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:44,552 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:44,552 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,553 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,553 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:44,554 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,554 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:44,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:44,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:44,563 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:44,563 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:44,563 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:44,563 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,563 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,566 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,565 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,566 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,567 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,569 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:44,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,571 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:44,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,571 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,572 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:44,573 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,573 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:44,573 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,573 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,573 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,672 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:44,673 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,673 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:44,684 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,685 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:44,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:44,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,687 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:44,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:44,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:44,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,687 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,688 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,688 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,689 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:44,689 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:44,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,696 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,700 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:44,700 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:44,700 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:44,700 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:44,700 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:44,714 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/482be5dcb70a4411bd2e1236ba86dab2 2023-02-08 08:55:45,126 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/2a679df4047f4f60a999f680cd2616f9 2023-02-08 08:55:45,130 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/482be5dcb70a4411bd2e1236ba86dab2 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2 2023-02-08 08:55:45,133 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:45,133 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/2a679df4047f4f60a999f680cd2616f9 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9 2023-02-08 08:55:45,136 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:45,137 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 437ms, sequenceid=32, compaction requested=true 2023-02-08 08:55:45,137 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:45,137 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,137 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,138 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,138 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:45,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@660afc99[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:45,140 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:45,140 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,141 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,142 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,142 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:45,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:45,142 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,146 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:45,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:45,149 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:45,149 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:45,149 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:45,149 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,149 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,151 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,152 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,152 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,153 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,153 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,155 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,156 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,156 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:45,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,159 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:45,159 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,160 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,260 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:45,260 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,260 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:45,270 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,270 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:45,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:45,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,272 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,272 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,272 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:45,272 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:45,272 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,273 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,274 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,274 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,274 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:45,276 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,276 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:45,277 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:45,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,279 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,279 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,284 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/b3ab968dd65344bfbe38dbc36be07835 2023-02-08 08:55:45,295 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/c1abada17620453eb60c582f99863aad 2023-02-08 08:55:45,299 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/b3ab968dd65344bfbe38dbc36be07835 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835 2023-02-08 08:55:45,302 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:45,302 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/c1abada17620453eb60c582f99863aad as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad 2023-02-08 08:55:45,305 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:45,305 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 28ms, sequenceid=45, compaction requested=true 2023-02-08 08:55:45,306 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:45,306 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,306 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:45,306 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:45,306 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,306 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,306 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,307 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,307 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,307 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,307 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:45,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63d6ffa0[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:45,309 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:45,309 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,310 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,310 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,310 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,311 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:45,310 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:45,313 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,313 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,313 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,314 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,314 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:45,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:45,317 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:45,317 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:45,317 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:45,317 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,317 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,318 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,319 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,319 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,320 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,321 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,322 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,322 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,329 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,330 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,430 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:45,430 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,430 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:45,432 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:45,441 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,441 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:45,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:45,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,443 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,443 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,443 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:45,443 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:45,443 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,443 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,444 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,444 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,444 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,444 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,445 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,445 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,446 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,447 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,446 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:45,447 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,447 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,448 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,449 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,457 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/7de3ef5ae13b492793137ee53a022d79 2023-02-08 08:55:45,471 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/5e65ed5141324bc8907229f82d13a451 2023-02-08 08:55:45,474 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/7de3ef5ae13b492793137ee53a022d79 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79 2023-02-08 08:55:45,477 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:45,478 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/5e65ed5141324bc8907229f82d13a451 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451 2023-02-08 08:55:45,481 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:45,482 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 35ms, sequenceid=58, compaction requested=true 2023-02-08 08:55:45,482 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:45,482 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,482 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:45,482 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:45,482 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,482 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,482 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,485 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,485 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,485 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,485 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9ed28a3[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:45,486 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:45,486 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,488 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,488 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:45,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,488 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:45,489 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,489 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,489 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,490 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,490 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:45,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:45,493 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:45,493 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:45,493 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:45,493 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,493 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,495 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,496 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,496 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,499 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:45,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,501 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:45,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,502 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,602 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:45,602 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,602 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:45,612 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,612 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:45,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:45,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,614 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,614 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,614 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 08:55:45,614 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 08:55:45,614 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,614 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,615 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,615 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,615 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,615 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,616 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:45,617 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:45,617 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,618 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. started... 2023-02-08 08:55:45,618 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:45,618 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,620 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,620 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,620 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,621 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,621 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,625 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/04161bf8d0de470da3d33bed12141df1 2023-02-08 08:55:45,634 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/57a61b06c7c94a6499f5351c26307fde 2023-02-08 08:55:45,638 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/04161bf8d0de470da3d33bed12141df1 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1 2023-02-08 08:55:45,641 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:45,641 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/57a61b06c7c94a6499f5351c26307fde as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde 2023-02-08 08:55:45,645 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:45,646 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 28ms, sequenceid=71, compaction requested=true 2023-02-08 08:55:45,646 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1349470b3394f45184feff0caaa8a0f5: 2023-02-08 08:55:45,646 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 08:55:45,646 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 08:55:45,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,647 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 08:55:45,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21ceeb00[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:45,650 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 08:55:45,650 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,653 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,653 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,653 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,654 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 08:55:45,654 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 08:55:45,654 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:45,656 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,656 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,656 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 08:55:45,657 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,657 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:45,660 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 08:55:45,660 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:45,660 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:45,660 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 08:55:45,660 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,660 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,661 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,662 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,662 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,663 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:45,664 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,664 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,664 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,667 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 08:55:45,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:45,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:45,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,668 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:45,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,669 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:45,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:45,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:45,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:45,669 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 08:55:45,769 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:45,769 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:45,769 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:45,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:45,777 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,777 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=1349470b3394f45184feff0caaa8a0f5, daughterA=cc4f7b985aa6efa9e8b281f265b91282, daughterB=62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:45,778 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=1349470b3394f45184feff0caaa8a0f5, daughterA=cc4f7b985aa6efa9e8b281f265b91282, daughterB=62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:45,778 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=1349470b3394f45184feff0caaa8a0f5, daughterA=cc4f7b985aa6efa9e8b281f265b91282, daughterB=62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:45,778 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=1349470b3394f45184feff0caaa8a0f5, daughterA=cc4f7b985aa6efa9e8b281f265b91282, daughterB=62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:45,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, UNASSIGN}] 2023-02-08 08:55:45,780 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, UNASSIGN 2023-02-08 08:55:45,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=1349470b3394f45184feff0caaa8a0f5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:45,781 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846545781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846545781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846545781"}]},"ts":"1675846545781"} 2023-02-08 08:55:45,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 1349470b3394f45184feff0caaa8a0f5, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:45,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:45,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1349470b3394f45184feff0caaa8a0f5, disabling compactions & flushes 2023-02-08 08:55:45,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. after waiting 0 ms 2023-02-08 08:55:45,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1349470b3394f45184feff0caaa8a0f5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:55:45,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/9289baba72df4140bb6fa992cbfd76c7 2023-02-08 08:55:45,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/d869946b97a74d0091410c128f03001c 2023-02-08 08:55:45,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf1/9289baba72df4140bb6fa992cbfd76c7 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7 2023-02-08 08:55:45,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:45,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.tmp/cf2/d869946b97a74d0091410c128f03001c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c 2023-02-08 08:55:45,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:45,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 31ms, sequenceid=82, compaction requested=true 2023-02-08 08:55:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:55:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:45,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. 2023-02-08 08:55:45,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1349470b3394f45184feff0caaa8a0f5: Waiting for close lock at 1675846545933Running coprocessor pre-close hooks at 1675846545933Disabling compacts and flushes for region at 1675846545933Disabling writes for close at 1675846545933Obtaining lock to block concurrent updates at 1675846545934 (+1 ms)Preparing flush snapshotting stores in 1349470b3394f45184feff0caaa8a0f5 at 1675846545934Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846545934Flushing stores of testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. at 1675846545934Flushing 1349470b3394f45184feff0caaa8a0f5/cf1: creating writer at 1675846545936 (+2 ms)Flushing 1349470b3394f45184feff0caaa8a0f5/cf1: appending metadata at 1675846545938 (+2 ms)Flushing 1349470b3394f45184feff0caaa8a0f5/cf1: closing flushed file at 1675846545938Flushing 1349470b3394f45184feff0caaa8a0f5/cf2: creating writer at 1675846545947 (+9 ms)Flushing 1349470b3394f45184feff0caaa8a0f5/cf2: appending metadata at 1675846545948 (+1 ms)Flushing 1349470b3394f45184feff0caaa8a0f5/cf2: closing flushed file at 1675846545948Flushing 1349470b3394f45184feff0caaa8a0f5/cf1: reopening flushed file at 1675846545957 (+9 ms)Flushing 1349470b3394f45184feff0caaa8a0f5/cf2: reopening flushed file at 1675846545961 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1349470b3394f45184feff0caaa8a0f5 in 31ms, sequenceid=82, compaction requested=true at 1675846545965 (+4 ms)Writing region close event to WAL at 1675846545968 (+3 ms)Running coprocessor post-close hooks at 1675846545971 (+3 ms)Closed at 1675846545972 (+1 ms) 2023-02-08 08:55:45,974 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:45,974 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=1349470b3394f45184feff0caaa8a0f5, regionState=CLOSED 2023-02-08 08:55:45,975 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846545974"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846545974"}]},"ts":"1675846545974"} 2023-02-08 08:55:45,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-08 08:55:45,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 1349470b3394f45184feff0caaa8a0f5, server=jenkins-hbase4.apache.org,44651,1675846441425 in 194 msec 2023-02-08 08:55:45,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-08 08:55:45,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=1349470b3394f45184feff0caaa8a0f5, UNASSIGN in 198 msec 2023-02-08 08:55:45,993 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=1349470b3394f45184feff0caaa8a0f5, threads=14 2023-02-08 08:55:45,993 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:45,993 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:45,993 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:45,996 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,000 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,004 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,004 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,004 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,008 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,011 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451, top=false 2023-02-08 08:55:46,011 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79, top=false 2023-02-08 08:55:46,013 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61, top=false 2023-02-08 08:55:46,016 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde, top=false 2023-02-08 08:55:46,019 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08, top=false 2023-02-08 08:55:46,019 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9, top=false 2023-02-08 08:55:46,019 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad, top=false 2023-02-08 08:55:46,020 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1, top=false 2023-02-08 08:55:46,021 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c, top=false 2023-02-08 08:55:46,021 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0, top=false 2023-02-08 08:55:46,021 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7, top=false 2023-02-08 08:55:46,021 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2, top=false 2023-02-08 08:55:46,025 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835, top=false 2023-02-08 08:55:46,025 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,027 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be, top=false 2023-02-08 08:55:46,031 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,032 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,032 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,033 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,033 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,037 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,037 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,038 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,039 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,040 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,042 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,047 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,047 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,047 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,047 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,047 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,048 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,048 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,050 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,051 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,051 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,051 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/.splits/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1 for child: cc4f7b985aa6efa9e8b281f265b91282, parent: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,052 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,053 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,054 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,054 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,056 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835 for region: 1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,056 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 1349470b3394f45184feff0caaa8a0f5 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-08 08:55:46,079 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:46,080 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:46,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675846546082"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675846546082"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675846546082"}]},"ts":"1675846546082"} 2023-02-08 08:55:46,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546082"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846546082"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846546082"}]},"ts":"1675846546082"} 2023-02-08 08:55:46,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546082"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846546082"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846546082"}]},"ts":"1675846546082"} 2023-02-08 08:55:46,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, ASSIGN}] 2023-02-08 08:55:46,085 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, ASSIGN 2023-02-08 08:55:46,085 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:46,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, ASSIGN 2023-02-08 08:55:46,086 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:46,236 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:55:46,237 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=62728f5a22fbc08be32697ce528ee57c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846546237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846546237"}]},"ts":"1675846546237"} 2023-02-08 08:55:46,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 62728f5a22fbc08be32697ce528ee57c, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:46,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=cc4f7b985aa6efa9e8b281f265b91282, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,241 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546241"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846546241"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846546241"}]},"ts":"1675846546241"} 2023-02-08 08:55:46,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure cc4f7b985aa6efa9e8b281f265b91282, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:46,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cc4f7b985aa6efa9e8b281f265b91282, NAME => 'testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.', STARTKEY => '', ENDKEY => '71'} 2023-02-08 08:55:46,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:46,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:46,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,395 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,396 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1 2023-02-08 08:55:46,396 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1 2023-02-08 08:55:46,397 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc4f7b985aa6efa9e8b281f265b91282 columnFamilyName cf1 2023-02-08 08:55:46,402 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1 2023-02-08 08:55:46,405 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2 2023-02-08 08:55:46,410 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79 2023-02-08 08:55:46,414 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61 2023-02-08 08:55:46,417 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7 2023-02-08 08:55:46,420 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835 2023-02-08 08:55:46,427 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be 2023-02-08 08:55:46,427 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(310): Store=cc4f7b985aa6efa9e8b281f265b91282/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:46,428 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,428 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2 2023-02-08 08:55:46,428 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2 2023-02-08 08:55:46,429 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc4f7b985aa6efa9e8b281f265b91282 columnFamilyName cf2 2023-02-08 08:55:46,433 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9 2023-02-08 08:55:46,436 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde 2023-02-08 08:55:46,439 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451 2023-02-08 08:55:46,441 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08 2023-02-08 08:55:46,444 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0 2023-02-08 08:55:46,447 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad 2023-02-08 08:55:46,449 DEBUG [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c 2023-02-08 08:55:46,449 INFO [StoreOpener-cc4f7b985aa6efa9e8b281f265b91282-1] regionserver.HStore(310): Store=cc4f7b985aa6efa9e8b281f265b91282/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:46,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:46,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cc4f7b985aa6efa9e8b281f265b91282; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62630726, jitterRate=-0.06672945618629456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:46,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cc4f7b985aa6efa9e8b281f265b91282: Running coprocessor pre-open hook at 1675846546394Writing region info on filesystem at 1675846546394Initializing all the Stores at 1675846546395 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846546395Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846546395Cleaning up temporary data from old regions at 1675846546451 (+56 ms)Cleaning up detritus from prior splits at 1675846546451Running coprocessor post-open hooks at 1675846546453 (+2 ms)Region opened successfully at 1675846546453 2023-02-08 08:55:46,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282., pid=134, masterSystemTime=1675846546390 2023-02-08 08:55:46,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:46,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): cc4f7b985aa6efa9e8b281f265b91282/cf2 is initiating minor compaction (all files) 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): cc4f7b985aa6efa9e8b281f265b91282/cf1 is initiating minor compaction (all files) 2023-02-08 08:55:46,455 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of cc4f7b985aa6efa9e8b281f265b91282/cf2 in testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,455 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cc4f7b985aa6efa9e8b281f265b91282/cf1 in testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,455 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/.tmp, totalSize=34.9 K 2023-02-08 08:55:46,455 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/.tmp, totalSize=34.9 K 2023-02-08 08:55:46,455 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846543937 2023-02-08 08:55:46,456 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846544100 2023-02-08 08:55:46,456 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846543937 2023-02-08 08:55:46,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 62728f5a22fbc08be32697ce528ee57c, NAME => 'testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.', STARTKEY => '71', ENDKEY => ''} 2023-02-08 08:55:46,456 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846544674 2023-02-08 08:55:46,456 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846544100 2023-02-08 08:55:46,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:46,457 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846545261 2023-02-08 08:55:46,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:46,457 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846544674 2023-02-08 08:55:46,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,457 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846545431 2023-02-08 08:55:46,457 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=cc4f7b985aa6efa9e8b281f265b91282, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,457 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546457"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846546457"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846546457"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846546457"}]},"ts":"1675846546457"} 2023-02-08 08:55:46,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-08 08:55:46,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure cc4f7b985aa6efa9e8b281f265b91282, server=jenkins-hbase4.apache.org,44651,1675846441425 in 216 msec 2023-02-08 08:55:46,460 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846545261 2023-02-08 08:55:46,460 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846545603 2023-02-08 08:55:46,460 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846545431 2023-02-08 08:55:46,461 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846545770 2023-02-08 08:55:46,461 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846545603 2023-02-08 08:55:46,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, ASSIGN in 376 msec 2023-02-08 08:55:46,461 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,461 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846545770 2023-02-08 08:55:46,464 DEBUG [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf1 2023-02-08 08:55:46,464 DEBUG [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf1 2023-02-08 08:55:46,464 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 62728f5a22fbc08be32697ce528ee57c columnFamilyName cf1 2023-02-08 08:55:46,465 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] regionserver.HStore(310): Store=62728f5a22fbc08be32697ce528ee57c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:46,465 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,466 DEBUG [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf2 2023-02-08 08:55:46,466 DEBUG [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf2 2023-02-08 08:55:46,466 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 62728f5a22fbc08be32697ce528ee57c columnFamilyName cf2 2023-02-08 08:55:46,467 INFO [StoreOpener-62728f5a22fbc08be32697ce528ee57c-1] regionserver.HStore(310): Store=62728f5a22fbc08be32697ce528ee57c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:46,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:46,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 62728f5a22fbc08be32697ce528ee57c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68903635, jitterRate=0.026744171977043152}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:46,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 62728f5a22fbc08be32697ce528ee57c: Running coprocessor pre-open hook at 1675846546457Writing region info on filesystem at 1675846546457Initializing all the Stores at 1675846546461 (+4 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846546461Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846546461Cleaning up temporary data from old regions at 1675846546469 (+8 ms)Cleaning up detritus from prior splits at 1675846546470 (+1 ms)Running coprocessor post-open hooks at 1675846546473 (+3 ms)Region opened successfully at 1675846546473 2023-02-08 08:55:46,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c., pid=133, masterSystemTime=1675846546390 2023-02-08 08:55:46,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=62728f5a22fbc08be32697ce528ee57c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546475"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846546475"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846546475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846546475"}]},"ts":"1675846546475"} 2023-02-08 08:55:46,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-08 08:55:46,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 62728f5a22fbc08be32697ce528ee57c, server=jenkins-hbase4.apache.org,44651,1675846441425 in 238 msec 2023-02-08 08:55:46,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-08 08:55:46,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, ASSIGN in 395 msec 2023-02-08 08:55:46,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=1349470b3394f45184feff0caaa8a0f5, daughterA=cc4f7b985aa6efa9e8b281f265b91282, daughterB=62728f5a22fbc08be32697ce528ee57c in 703 msec 2023-02-08 08:55:46,484 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): cc4f7b985aa6efa9e8b281f265b91282#cf1#compaction#124 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:46,488 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): cc4f7b985aa6efa9e8b281f265b91282#cf2#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:46,502 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/.tmp/cf1/6e29c352ff324b3597ef82d86a135521 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/6e29c352ff324b3597ef82d86a135521 2023-02-08 08:55:46,503 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/.tmp/cf2/a4c8b32fa10c494fbb69e1241ea612c2 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/a4c8b32fa10c494fbb69e1241ea612c2 2023-02-08 08:55:46,506 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cc4f7b985aa6efa9e8b281f265b91282/cf1 of cc4f7b985aa6efa9e8b281f265b91282 into 6e29c352ff324b3597ef82d86a135521(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:46,506 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cc4f7b985aa6efa9e8b281f265b91282: 2023-02-08 08:55:46,506 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282., storeName=cc4f7b985aa6efa9e8b281f265b91282/cf1, priority=9, startTime=1675846546454; duration=0sec 2023-02-08 08:55:46,506 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:46,507 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cc4f7b985aa6efa9e8b281f265b91282/cf2 of cc4f7b985aa6efa9e8b281f265b91282 into a4c8b32fa10c494fbb69e1241ea612c2(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:46,507 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for cc4f7b985aa6efa9e8b281f265b91282: 2023-02-08 08:55:46,507 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282., storeName=cc4f7b985aa6efa9e8b281f265b91282/cf2, priority=9, startTime=1675846546455; duration=0sec 2023-02-08 08:55:46,507 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:46,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44651] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:49624 deadline: 1675846606528, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5. is not online on jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,637 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-08 08:55:46,640 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:46,640 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,640 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,641 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,641 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,641 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,644 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,644 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 08:55:46,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846546644"}]},"ts":"1675846546644"} 2023-02-08 08:55:46,645 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-08 08:55:46,647 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-08 08:55:46,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, UNASSIGN}] 2023-02-08 08:55:46,648 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, UNASSIGN 2023-02-08 08:55:46,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=62728f5a22fbc08be32697ce528ee57c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546648"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846546648"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846546648"}]},"ts":"1675846546648"} 2023-02-08 08:55:46,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 62728f5a22fbc08be32697ce528ee57c, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:46,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, UNASSIGN 2023-02-08 08:55:46,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=cc4f7b985aa6efa9e8b281f265b91282, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:46,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846546651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846546651"}]},"ts":"1675846546651"} 2023-02-08 08:55:46,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure cc4f7b985aa6efa9e8b281f265b91282, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:46,745 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,745 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 08:55:46,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cc4f7b985aa6efa9e8b281f265b91282, disabling compactions & flushes 2023-02-08 08:55:46,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. after waiting 0 ms 2023-02-08 08:55:46,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,801 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7] to archive 2023-02-08 08:55:46,802 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:46,803 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1124596cd0e4d34903f6115ee5334be 2023-02-08 08:55:46,804 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-873b1a56a3a4438283e849d1edfe8b61 2023-02-08 08:55:46,805 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-482be5dcb70a4411bd2e1236ba86dab2 2023-02-08 08:55:46,806 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b3ab968dd65344bfbe38dbc36be07835 2023-02-08 08:55:46,807 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-7de3ef5ae13b492793137ee53a022d79 2023-02-08 08:55:46,808 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-04161bf8d0de470da3d33bed12141df1 2023-02-08 08:55:46,808 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-9289baba72df4140bb6fa992cbfd76c7 2023-02-08 08:55:46,810 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c] to archive 2023-02-08 08:55:46,810 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:46,811 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-bdd94c8cebc84b5ab1b4efe400b7b1b0 2023-02-08 08:55:46,812 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-b8cce518b1274ce8829539280a33af08 2023-02-08 08:55:46,813 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-2a679df4047f4f60a999f680cd2616f9 2023-02-08 08:55:46,814 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-c1abada17620453eb60c582f99863aad 2023-02-08 08:55:46,815 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-5e65ed5141324bc8907229f82d13a451 2023-02-08 08:55:46,817 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-57a61b06c7c94a6499f5351c26307fde 2023-02-08 08:55:46,818 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/testSplitTableRegionEmptyDaughter=1349470b3394f45184feff0caaa8a0f5-d869946b97a74d0091410c128f03001c 2023-02-08 08:55:46,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:55:46,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:46,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282. 2023-02-08 08:55:46,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cc4f7b985aa6efa9e8b281f265b91282: Waiting for close lock at 1675846546801Running coprocessor pre-close hooks at 1675846546801Disabling compacts and flushes for region at 1675846546801Disabling writes for close at 1675846546801Writing region close event to WAL at 1675846546818 (+17 ms)Running coprocessor post-close hooks at 1675846546821 (+3 ms)Closed at 1675846546821 2023-02-08 08:55:46,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 62728f5a22fbc08be32697ce528ee57c, disabling compactions & flushes 2023-02-08 08:55:46,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. after waiting 0 ms 2023-02-08 08:55:46,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=cc4f7b985aa6efa9e8b281f265b91282, regionState=CLOSED 2023-02-08 08:55:46,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546824"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846546824"}]},"ts":"1675846546824"} 2023-02-08 08:55:46,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-08 08:55:46,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-08 08:55:46,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure cc4f7b985aa6efa9e8b281f265b91282, server=jenkins-hbase4.apache.org,44651,1675846441425 in 173 msec 2023-02-08 08:55:46,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:46,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c. 2023-02-08 08:55:46,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 62728f5a22fbc08be32697ce528ee57c: Waiting for close lock at 1675846546823Running coprocessor pre-close hooks at 1675846546823Disabling compacts and flushes for region at 1675846546823Disabling writes for close at 1675846546824 (+1 ms)Writing region close event to WAL at 1675846546824Running coprocessor post-close hooks at 1675846546827 (+3 ms)Closed at 1675846546827 2023-02-08 08:55:46,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cc4f7b985aa6efa9e8b281f265b91282, UNASSIGN in 180 msec 2023-02-08 08:55:46,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=62728f5a22fbc08be32697ce528ee57c, regionState=CLOSED 2023-02-08 08:55:46,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846546829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846546829"}]},"ts":"1675846546829"} 2023-02-08 08:55:46,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-08 08:55:46,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 62728f5a22fbc08be32697ce528ee57c, server=jenkins-hbase4.apache.org,44651,1675846441425 in 181 msec 2023-02-08 08:55:46,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-08 08:55:46,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=62728f5a22fbc08be32697ce528ee57c, UNASSIGN in 184 msec 2023-02-08 08:55:46,833 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846546833"}]},"ts":"1675846546833"} 2023-02-08 08:55:46,834 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-08 08:55:46,835 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-08 08:55:46,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 193 msec 2023-02-08 08:55:46,946 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,946 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 08:55:46,947 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-08 08:55:46,947 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,947 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,950 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,950 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 08:55:46,954 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,954 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,954 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,956 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/recovered.edits] 2023-02-08 08:55:46,956 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/recovered.edits] 2023-02-08 08:55:46,956 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/recovered.edits] 2023-02-08 08:55:46,958 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/04161bf8d0de470da3d33bed12141df1 2023-02-08 08:55:46,959 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/6e29c352ff324b3597ef82d86a135521 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf1/6e29c352ff324b3597ef82d86a135521 2023-02-08 08:55:46,960 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/482be5dcb70a4411bd2e1236ba86dab2 2023-02-08 08:55:46,961 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/recovered.edits/88.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c/recovered.edits/88.seqid 2023-02-08 08:55:46,961 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/7de3ef5ae13b492793137ee53a022d79 2023-02-08 08:55:46,961 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/a4c8b32fa10c494fbb69e1241ea612c2 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/cf2/a4c8b32fa10c494fbb69e1241ea612c2 2023-02-08 08:55:46,961 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/62728f5a22fbc08be32697ce528ee57c 2023-02-08 08:55:46,962 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/873b1a56a3a4438283e849d1edfe8b61 2023-02-08 08:55:46,963 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/9289baba72df4140bb6fa992cbfd76c7 2023-02-08 08:55:46,964 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282/recovered.edits/90.seqid 2023-02-08 08:55:46,964 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/cc4f7b985aa6efa9e8b281f265b91282 2023-02-08 08:55:46,965 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/b3ab968dd65344bfbe38dbc36be07835 2023-02-08 08:55:46,965 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf1/c1124596cd0e4d34903f6115ee5334be 2023-02-08 08:55:46,967 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/2a679df4047f4f60a999f680cd2616f9 2023-02-08 08:55:46,968 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/57a61b06c7c94a6499f5351c26307fde 2023-02-08 08:55:46,969 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/5e65ed5141324bc8907229f82d13a451 2023-02-08 08:55:46,970 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/b8cce518b1274ce8829539280a33af08 2023-02-08 08:55:46,970 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/bdd94c8cebc84b5ab1b4efe400b7b1b0 2023-02-08 08:55:46,971 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/c1abada17620453eb60c582f99863aad 2023-02-08 08:55:46,972 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/cf2/d869946b97a74d0091410c128f03001c 2023-02-08 08:55:46,974 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5/recovered.edits/85.seqid 2023-02-08 08:55:46,974 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegionEmptyDaughter/1349470b3394f45184feff0caaa8a0f5 2023-02-08 08:55:46,974 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 08:55:46,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,977 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-08 08:55:46,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-08 08:55:46,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-08 08:55:46,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846546979"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:46,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846546979"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:46,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846546979"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:46,981 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:55:46,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1349470b3394f45184feff0caaa8a0f5, NAME => 'testSplitTableRegionEmptyDaughter,,1675846543434.1349470b3394f45184feff0caaa8a0f5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cc4f7b985aa6efa9e8b281f265b91282, NAME => 'testSplitTableRegionEmptyDaughter,,1675846545777.cc4f7b985aa6efa9e8b281f265b91282.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 62728f5a22fbc08be32697ce528ee57c, NAME => 'testSplitTableRegionEmptyDaughter,71,1675846545777.62728f5a22fbc08be32697ce528ee57c.', STARTKEY => '71', ENDKEY => ''}] 2023-02-08 08:55:46,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-08 08:55:46,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846546981"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:46,982 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-08 08:55:46,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 08:55:46,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 35 msec 2023-02-08 08:55:47,051 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:47,051 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:47,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 08:55:47,052 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-08 08:55:47,065 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=371 (was 371), OpenFileDescriptor=835 (was 833) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=130 (was 133), ProcessCount=168 (was 168), AvailableMemoryMB=3974 (was 3978) 2023-02-08 08:55:47,078 INFO [Listener at localhost/33105] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=371, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=130, ProcessCount=168, AvailableMemoryMB=3973 2023-02-08 08:55:47,078 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:47,078 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:47,078 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:47,081 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 08:55:47,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-08 08:55:47,082 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-08 08:55:47,084 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:47,084 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:47,086 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,086 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 empty. 2023-02-08 08:55:47,086 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,086 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 08:55:47,094 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:47,095 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 30132006af7f58dd501907fc8f96d2f7, NAME => 'testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 30132006af7f58dd501907fc8f96d2f7, disabling compactions & flushes 2023-02-08 08:55:47,101 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. after waiting 0 ms 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,101 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,101 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 30132006af7f58dd501907fc8f96d2f7: Waiting for close lock at 1675846547101Disabling compacts and flushes for region at 1675846547101Disabling writes for close at 1675846547101Writing region close event to WAL at 1675846547101Closed at 1675846547101 2023-02-08 08:55:47,103 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:47,103 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846547103"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846547103"}]},"ts":"1675846547103"} 2023-02-08 08:55:47,104 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:47,105 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:47,105 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846547105"}]},"ts":"1675846547105"} 2023-02-08 08:55:47,106 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-08 08:55:47,108 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-08 08:55:47,108 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 08:55:47,109 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 08:55:47,109 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 08:55:47,109 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 08:55:47,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, ASSIGN}] 2023-02-08 08:55:47,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, ASSIGN 2023-02-08 08:55:47,110 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:47,260 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 08:55:47,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=30132006af7f58dd501907fc8f96d2f7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846547262"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846547262"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846547262"}]},"ts":"1675846547262"} 2023-02-08 08:55:47,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 30132006af7f58dd501907fc8f96d2f7, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:47,373 ERROR [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 260f0410-1e75-4d9d-aa4e-3ba6e41b0476) service to localhost/127.0.0.1:42159 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:55:47,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 30132006af7f58dd501907fc8f96d2f7, NAME => 'testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:47,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,418 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,419 DEBUG [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1 2023-02-08 08:55:47,419 DEBUG [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1 2023-02-08 08:55:47,420 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30132006af7f58dd501907fc8f96d2f7 columnFamilyName cf1 2023-02-08 08:55:47,420 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] regionserver.HStore(310): Store=30132006af7f58dd501907fc8f96d2f7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:47,420 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,421 DEBUG [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2 2023-02-08 08:55:47,421 DEBUG [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2 2023-02-08 08:55:47,422 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30132006af7f58dd501907fc8f96d2f7 columnFamilyName cf2 2023-02-08 08:55:47,422 INFO [StoreOpener-30132006af7f58dd501907fc8f96d2f7-1] regionserver.HStore(310): Store=30132006af7f58dd501907fc8f96d2f7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:47,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:47,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:47,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:47,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 30132006af7f58dd501907fc8f96d2f7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70878868, jitterRate=0.05617743730545044}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:47,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 30132006af7f58dd501907fc8f96d2f7: Running coprocessor pre-open hook at 1675846547417Writing region info on filesystem at 1675846547417Initializing all the Stores at 1675846547418 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846547418Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846547418Cleaning up temporary data from old regions at 1675846547423 (+5 ms)Cleaning up detritus from prior splits at 1675846547424 (+1 ms)Running coprocessor post-open hooks at 1675846547426 (+2 ms)Region opened successfully at 1675846547426 2023-02-08 08:55:47,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7., pid=143, masterSystemTime=1675846547414 2023-02-08 08:55:47,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,428 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=30132006af7f58dd501907fc8f96d2f7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846547428"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846547428"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846547428"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846547428"}]},"ts":"1675846547428"} 2023-02-08 08:55:47,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-08 08:55:47,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 30132006af7f58dd501907fc8f96d2f7, server=jenkins-hbase4.apache.org,44651,1675846441425 in 167 msec 2023-02-08 08:55:47,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-08 08:55:47,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, ASSIGN in 322 msec 2023-02-08 08:55:47,432 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:47,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846547432"}]},"ts":"1675846547432"} 2023-02-08 08:55:47,433 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-08 08:55:47,435 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:47,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 353 msec 2023-02-08 08:55:47,589 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:47,589 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:47,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:47,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:47,590 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:47,591 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,591 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:47,591 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:47,591 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,591 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,593 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,593 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,593 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,593 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,594 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,594 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:47,595 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:47,596 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:47,596 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,596 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:47,596 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:47,596 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:47,597 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 08:55:47,598 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,598 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,598 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,598 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:47,598 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,599 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:47,605 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/bb4da817ed814fe288b4e4f770d44116 2023-02-08 08:55:47,615 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/0fc62344523b4c5796d037078428058e 2023-02-08 08:55:47,618 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/bb4da817ed814fe288b4e4f770d44116 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116 2023-02-08 08:55:47,621 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:47,621 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/0fc62344523b4c5796d037078428058e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e 2023-02-08 08:55:47,624 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:47,625 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 28ms, sequenceid=6, compaction requested=false 2023-02-08 08:55:47,625 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-08 08:55:47,626 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:47,626 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,626 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:47,626 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:47,626 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,626 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:47,626 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:47,627 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,627 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:47,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:47,627 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,628 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:47,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:47,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:47,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:47,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:47,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10113594[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:47,629 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:47,629 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,631 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:47,631 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:47,632 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:47,633 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:47,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:47,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:47,633 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:47,637 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:47,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:47,637 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:47,637 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:47,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:47,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:47,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:47,639 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:47,639 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:47,639 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:47,639 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:47,639 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:47,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,641 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,642 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,642 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,644 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,644 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:47,644 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,646 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:47,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:47,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:47,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,648 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,647 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:47,648 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:47,648 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,648 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,648 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,748 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:47,748 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:47,748 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:47,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:47,759 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:47,759 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:47,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:47,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:47,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:47,761 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:47,761 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,761 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:47,761 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:47,761 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,761 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,762 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,762 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,762 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,762 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:47,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:47,763 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:47,763 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:47,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:47,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:47,765 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:47,765 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:47,765 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:47,766 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,767 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,767 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,767 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:47,768 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:47,768 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:47,773 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/4d00b2fa5a0a404f859579dcc305b75c 2023-02-08 08:55:48,184 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/ca7f56b3389a4a15ba54ba30e90143a0 2023-02-08 08:55:48,188 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/4d00b2fa5a0a404f859579dcc305b75c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c 2023-02-08 08:55:48,191 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:48,192 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/ca7f56b3389a4a15ba54ba30e90143a0 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0 2023-02-08 08:55:48,195 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 08:55:48,195 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 430ms, sequenceid=19, compaction requested=false 2023-02-08 08:55:48,195 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:48,195 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,195 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,197 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44eee3c9[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:48,199 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:48,199 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,200 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:48,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,201 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,203 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,203 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,203 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,203 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,204 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,203 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,204 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:48,206 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:48,207 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:48,207 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:48,207 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:48,207 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,208 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,208 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,209 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,209 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,211 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:48,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,213 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:48,213 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,214 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,314 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:48,314 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,314 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:48,323 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,323 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:48,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:48,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:48,325 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:48,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:48,325 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:48,326 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,326 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,328 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,329 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,329 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,330 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:48,331 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,331 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:48,331 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,334 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,334 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,334 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,334 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,342 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/42a0b2719f7a4f969ebe1fe4403588c7 2023-02-08 08:55:48,351 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/41f2f112068b48b2b678f5b7cd11e1f6 2023-02-08 08:55:48,354 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/42a0b2719f7a4f969ebe1fe4403588c7 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7 2023-02-08 08:55:48,357 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:48,358 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/41f2f112068b48b2b678f5b7cd11e1f6 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6 2023-02-08 08:55:48,361 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 08:55:48,362 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 31ms, sequenceid=32, compaction requested=true 2023-02-08 08:55:48,362 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:48,362 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,365 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:48,367 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a663c7[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:48,367 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,368 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,368 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,369 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:48,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,368 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,371 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,371 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,371 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,373 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,373 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,373 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,373 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:48,376 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:48,376 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:48,376 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:48,376 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:48,377 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 ERROR [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f) service to localhost/127.0.0.1:42159 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:55:48,378 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,379 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,379 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,380 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,381 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,382 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,382 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:48,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,384 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,384 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,385 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,386 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,486 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:48,486 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,486 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:48,495 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,495 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:48,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:48,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:48,497 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:48,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:48,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:48,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,497 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,498 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,498 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,498 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,498 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,499 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,499 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,500 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,501 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,501 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,501 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,501 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:48,501 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:48,501 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,501 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:48,501 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:48,503 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,503 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,513 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/371a56ada5964654930bef60194dd292 2023-02-08 08:55:48,526 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/d6e8ea64755d4d7bbc1d63643878c47a 2023-02-08 08:55:48,529 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/371a56ada5964654930bef60194dd292 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292 2023-02-08 08:55:48,532 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:48,533 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/d6e8ea64755d4d7bbc1d63643878c47a as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a 2023-02-08 08:55:48,535 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 08:55:48,536 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 35ms, sequenceid=45, compaction requested=true 2023-02-08 08:55:48,536 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:48,536 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,536 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:48,536 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:48,537 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,537 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,537 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,538 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,538 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,538 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,538 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:48,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72661ab8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:48,540 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:48,540 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:48,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,541 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:48,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,542 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:48,541 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,543 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,545 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,545 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,545 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:48,548 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:48,548 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:48,548 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:48,548 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:48,548 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,550 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,551 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,552 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,552 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,554 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,554 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,554 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:48,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,559 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,560 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:48,559 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,560 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,560 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:48,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,560 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,561 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,561 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,561 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,561 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,561 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,660 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:48,661 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,661 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:48,670 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,670 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:48,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:48,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:48,672 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:48,672 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,672 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:48,672 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:48,672 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,672 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,674 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,674 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,674 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:48,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,675 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,676 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,676 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,676 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,676 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:48,676 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,676 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,676 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,676 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:48,677 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,678 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,678 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,678 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,678 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,678 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,684 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/927e2f66abc64f339e7a471b509c8895 2023-02-08 08:55:48,700 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/3aca5001c3394b17bd930d7c618a992d 2023-02-08 08:55:48,703 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/927e2f66abc64f339e7a471b509c8895 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895 2023-02-08 08:55:48,706 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:48,707 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/3aca5001c3394b17bd930d7c618a992d as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d 2023-02-08 08:55:48,709 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 08:55:48,710 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 34ms, sequenceid=58, compaction requested=true 2023-02-08 08:55:48,710 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:48,710 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,712 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,712 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,712 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,712 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:48,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b423cf5[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:48,714 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:48,714 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:48,716 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,716 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:48,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:48,716 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:48,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,717 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,718 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,719 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,719 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,719 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,718 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,719 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,719 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:48,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:48,722 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:48,722 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:48,722 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:48,722 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:48,722 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,723 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,724 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,725 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,726 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,726 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,727 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:48,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:48,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,730 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,730 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:48,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,731 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,831 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:48,831 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,831 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:48,841 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:48,841 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:48,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:48,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:48,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 08:55:48,843 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 08:55:48,843 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,843 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 08:55:48,843 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 08:55:48,844 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,844 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,845 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,845 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,845 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:48,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,846 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 08:55:48,846 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 08:55:48,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36805,1675846441472' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39055,1675846441504' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:48,847 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:48,847 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,847 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. started... 2023-02-08 08:55:48,848 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 08:55:48,848 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,849 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,849 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,849 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,849 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,849 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,857 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/a3b28368bd994b05b5d7c13f16bdec77 2023-02-08 08:55:48,866 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/896e3fb926c94ef6b8fa1bd9650d167c 2023-02-08 08:55:48,869 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/a3b28368bd994b05b5d7c13f16bdec77 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77 2023-02-08 08:55:48,872 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:48,873 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/896e3fb926c94ef6b8fa1bd9650d167c as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c 2023-02-08 08:55:48,876 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 08:55:48,877 INFO [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 30ms, sequenceid=71, compaction requested=true 2023-02-08 08:55:48,877 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 30132006af7f58dd501907fc8f96d2f7: 2023-02-08 08:55:48,877 DEBUG [rs(jenkins-hbase4.apache.org,44651,1675846441425)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:48,877 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:48,877 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:48,877 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,877 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 08:55:48,877 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 08:55:48,879 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,879 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,879 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,880 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:48,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44651,1675846441425' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 08:55:48,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58017aff[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:48,881 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 08:55:48,881 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,882 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,882 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,882 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,883 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,883 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39055,1675846441504' in zk 2023-02-08 08:55:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36805,1675846441472' in zk 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 08:55:48,883 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44651,1675846441425' in zk 2023-02-08 08:55:48,887 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,887 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,887 DEBUG [member: 'jenkins-hbase4.apache.org,39055,1675846441504' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 08:55:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,888 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,889 DEBUG [member: 'jenkins-hbase4.apache.org,44651,1675846441425' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,889 DEBUG [member: 'jenkins-hbase4.apache.org,36805,1675846441472' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,44651,1675846441425': 2023-02-08 08:55:48,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44651,1675846441425' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 08:55:48,891 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 08:55:48,892 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:48,892 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:48,892 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 08:55:48,892 INFO [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,893 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,893 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:48,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:48,894 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,895 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:48,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:48,896 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 08:55:48,896 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,897 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,899 DEBUG [(jenkins-hbase4.apache.org,33127,1675846440297)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:48,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 08:55:48,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:48,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,899 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:48,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:48,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 08:55:48,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:48,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:48,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 08:55:48,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:48,900 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:48,901 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,901 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:48,901 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:48,901 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 08:55:48,901 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 08:55:49,000 DEBUG [Listener at localhost/33105] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:49,001 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:49,001 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:49,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:49,010 DEBUG [Listener at localhost/33105] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:49,011 DEBUG [Listener at localhost/33105] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=30132006af7f58dd501907fc8f96d2f7, daughterA=4978a059e5d05283e1c29386375100b5, daughterB=46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:49,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=30132006af7f58dd501907fc8f96d2f7, daughterA=4978a059e5d05283e1c29386375100b5, daughterB=46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:49,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=30132006af7f58dd501907fc8f96d2f7, daughterA=4978a059e5d05283e1c29386375100b5, daughterB=46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:49,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=30132006af7f58dd501907fc8f96d2f7, daughterA=4978a059e5d05283e1c29386375100b5, daughterB=46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:49,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, UNASSIGN}] 2023-02-08 08:55:49,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, UNASSIGN 2023-02-08 08:55:49,016 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=30132006af7f58dd501907fc8f96d2f7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:49,016 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846549016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846549016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846549016"}]},"ts":"1675846549016"} 2023-02-08 08:55:49,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 30132006af7f58dd501907fc8f96d2f7, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:49,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 30132006af7f58dd501907fc8f96d2f7, disabling compactions & flushes 2023-02-08 08:55:49,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:49,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:49,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. after waiting 0 ms 2023-02-08 08:55:49,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:49,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 30132006af7f58dd501907fc8f96d2f7 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 08:55:49,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/ff22f0949ef046f29aa73d320be04e36 2023-02-08 08:55:49,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/ccf310d74d4d462c9bd662eaf7da7354 2023-02-08 08:55:49,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf1/ff22f0949ef046f29aa73d320be04e36 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36 2023-02-08 08:55:49,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:49,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/.tmp/cf2/ccf310d74d4d462c9bd662eaf7da7354 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354 2023-02-08 08:55:49,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 08:55:49,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 29ms, sequenceid=82, compaction requested=true 2023-02-08 08:55:49,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 08:55:49,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:49,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. 2023-02-08 08:55:49,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 30132006af7f58dd501907fc8f96d2f7: Waiting for close lock at 1675846549169Running coprocessor pre-close hooks at 1675846549169Disabling compacts and flushes for region at 1675846549169Disabling writes for close at 1675846549169Obtaining lock to block concurrent updates at 1675846549169Preparing flush snapshotting stores in 30132006af7f58dd501907fc8f96d2f7 at 1675846549169Finished memstore snapshotting testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675846549170 (+1 ms)Flushing stores of testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. at 1675846549170Flushing 30132006af7f58dd501907fc8f96d2f7/cf1: creating writer at 1675846549171 (+1 ms)Flushing 30132006af7f58dd501907fc8f96d2f7/cf1: appending metadata at 1675846549173 (+2 ms)Flushing 30132006af7f58dd501907fc8f96d2f7/cf1: closing flushed file at 1675846549173Flushing 30132006af7f58dd501907fc8f96d2f7/cf2: creating writer at 1675846549182 (+9 ms)Flushing 30132006af7f58dd501907fc8f96d2f7/cf2: appending metadata at 1675846549183 (+1 ms)Flushing 30132006af7f58dd501907fc8f96d2f7/cf2: closing flushed file at 1675846549183Flushing 30132006af7f58dd501907fc8f96d2f7/cf1: reopening flushed file at 1675846549192 (+9 ms)Flushing 30132006af7f58dd501907fc8f96d2f7/cf2: reopening flushed file at 1675846549195 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 30132006af7f58dd501907fc8f96d2f7 in 29ms, sequenceid=82, compaction requested=true at 1675846549198 (+3 ms)Writing region close event to WAL at 1675846549201 (+3 ms)Running coprocessor post-close hooks at 1675846549203 (+2 ms)Closed at 1675846549204 (+1 ms) 2023-02-08 08:55:49,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=30132006af7f58dd501907fc8f96d2f7, regionState=CLOSED 2023-02-08 08:55:49,205 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846549205"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846549205"}]},"ts":"1675846549205"} 2023-02-08 08:55:49,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-08 08:55:49,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 30132006af7f58dd501907fc8f96d2f7, server=jenkins-hbase4.apache.org,44651,1675846441425 in 189 msec 2023-02-08 08:55:49,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-08 08:55:49,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=30132006af7f58dd501907fc8f96d2f7, UNASSIGN in 193 msec 2023-02-08 08:55:49,217 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=30132006af7f58dd501907fc8f96d2f7, threads=14 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,217 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,218 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,367 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,367 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,367 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,384 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,384 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,387 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,391 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,392 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,392 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,393 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,736 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,736 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,738 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,760 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6 for region: 30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:49,761 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 30132006af7f58dd501907fc8f96d2f7 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 08:55:49,788 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:49,789 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 08:55:49,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675846549790"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675846549790"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675846549790"}]},"ts":"1675846549790"} 2023-02-08 08:55:49,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846549790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846549790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846549790"}]},"ts":"1675846549790"} 2023-02-08 08:55:49,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846549790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846549790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846549790"}]},"ts":"1675846549790"} 2023-02-08 08:55:49,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, ASSIGN}] 2023-02-08 08:55:49,793 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, ASSIGN 2023-02-08 08:55:49,793 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:49,794 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, ASSIGN 2023-02-08 08:55:49,794 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44651,1675846441425; forceNewPlan=false, retain=false 2023-02-08 08:55:49,944 INFO [jenkins-hbase4:33127] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 08:55:49,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=46131037d0eaa12d89cacb90c1b92e8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:49,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846549945"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846549945"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846549945"}]},"ts":"1675846549945"} 2023-02-08 08:55:49,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 46131037d0eaa12d89cacb90c1b92e8c, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:49,949 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=4978a059e5d05283e1c29386375100b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:49,949 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846549949"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846549949"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846549949"}]},"ts":"1675846549949"} 2023-02-08 08:55:49,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 4978a059e5d05283e1c29386375100b5, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:50,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 46131037d0eaa12d89cacb90c1b92e8c, NAME => 'testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 08:55:50,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:50,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:50,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,103 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,103 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1 2023-02-08 08:55:50,103 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1 2023-02-08 08:55:50,104 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46131037d0eaa12d89cacb90c1b92e8c columnFamilyName cf1 2023-02-08 08:55:50,114 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-top 2023-02-08 08:55:50,117 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-top 2023-02-08 08:55:50,120 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-top 2023-02-08 08:55:50,123 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-top 2023-02-08 08:55:50,125 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-top 2023-02-08 08:55:50,128 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-top 2023-02-08 08:55:50,131 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-top 2023-02-08 08:55:50,131 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(310): Store=46131037d0eaa12d89cacb90c1b92e8c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:50,131 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,132 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2 2023-02-08 08:55:50,132 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2 2023-02-08 08:55:50,132 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46131037d0eaa12d89cacb90c1b92e8c columnFamilyName cf2 2023-02-08 08:55:50,142 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-top 2023-02-08 08:55:50,145 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-top 2023-02-08 08:55:50,148 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-top 2023-02-08 08:55:50,151 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-top 2023-02-08 08:55:50,154 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-top 2023-02-08 08:55:50,157 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-top 2023-02-08 08:55:50,159 DEBUG [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-top 2023-02-08 08:55:50,160 INFO [StoreOpener-46131037d0eaa12d89cacb90c1b92e8c-1] regionserver.HStore(310): Store=46131037d0eaa12d89cacb90c1b92e8c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:50,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:50,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 46131037d0eaa12d89cacb90c1b92e8c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71080376, jitterRate=0.05918014049530029}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:50,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 46131037d0eaa12d89cacb90c1b92e8c: Running coprocessor pre-open hook at 1675846550102Writing region info on filesystem at 1675846550102Initializing all the Stores at 1675846550102Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846550102Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846550102Cleaning up temporary data from old regions at 1675846550162 (+60 ms)Cleaning up detritus from prior splits at 1675846550162Running coprocessor post-open hooks at 1675846550164 (+2 ms)Region opened successfully at 1675846550164 2023-02-08 08:55:50,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c., pid=149, masterSystemTime=1675846550098 2023-02-08 08:55:50,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,165 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:50,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,166 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:50,166 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): 46131037d0eaa12d89cacb90c1b92e8c/cf1 is initiating minor compaction (all files) 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 46131037d0eaa12d89cacb90c1b92e8c/cf1 in testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,166 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): 46131037d0eaa12d89cacb90c1b92e8c/cf2 is initiating minor compaction (all files) 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 46131037d0eaa12d89cacb90c1b92e8c/cf2 in testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/.tmp, totalSize=34.9 K 2023-02-08 08:55:50,166 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-top] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/.tmp, totalSize=34.9 K 2023-02-08 08:55:50,167 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846547585 2023-02-08 08:55:50,167 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675846547585 2023-02-08 08:55:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,167 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846547749 2023-02-08 08:55:50,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4978a059e5d05283e1c29386375100b5, NAME => 'testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 08:55:50,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 08:55:50,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:50,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,168 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675846547749 2023-02-08 08:55:50,168 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846548316 2023-02-08 08:55:50,168 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=46131037d0eaa12d89cacb90c1b92e8c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:50,168 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550168"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846550168"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846550168"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846550168"}]},"ts":"1675846550168"} 2023-02-08 08:55:50,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-08 08:55:50,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 46131037d0eaa12d89cacb90c1b92e8c, server=jenkins-hbase4.apache.org,44651,1675846441425 in 222 msec 2023-02-08 08:55:50,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, ASSIGN in 379 msec 2023-02-08 08:55:50,172 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675846548316 2023-02-08 08:55:50,172 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846548487 2023-02-08 08:55:50,173 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675846548487 2023-02-08 08:55:50,173 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,173 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846548662 2023-02-08 08:55:50,173 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675846548662 2023-02-08 08:55:50,173 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846548833 2023-02-08 08:55:50,174 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675846548833 2023-02-08 08:55:50,174 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846549002 2023-02-08 08:55:50,174 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1 2023-02-08 08:55:50,174 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675846549002 2023-02-08 08:55:50,174 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1 2023-02-08 08:55:50,175 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4978a059e5d05283e1c29386375100b5 columnFamilyName cf1 2023-02-08 08:55:50,188 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-bottom 2023-02-08 08:55:50,192 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46131037d0eaa12d89cacb90c1b92e8c#cf1#compaction#140 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:50,193 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): 46131037d0eaa12d89cacb90c1b92e8c#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:50,193 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-bottom 2023-02-08 08:55:50,197 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-bottom 2023-02-08 08:55:50,207 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-bottom 2023-02-08 08:55:50,211 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-bottom 2023-02-08 08:55:50,212 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/.tmp/cf1/358d69053326413bace24489487bb617 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/358d69053326413bace24489487bb617 2023-02-08 08:55:50,215 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-bottom 2023-02-08 08:55:50,218 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46131037d0eaa12d89cacb90c1b92e8c/cf1 of 46131037d0eaa12d89cacb90c1b92e8c into 358d69053326413bace24489487bb617(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:50,218 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46131037d0eaa12d89cacb90c1b92e8c: 2023-02-08 08:55:50,218 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c., storeName=46131037d0eaa12d89cacb90c1b92e8c/cf1, priority=9, startTime=1675846550165; duration=0sec 2023-02-08 08:55:50,218 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,219 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-bottom 2023-02-08 08:55:50,219 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(310): Store=4978a059e5d05283e1c29386375100b5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:50,219 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/.tmp/cf2/fc635e1c37354dad89e73e8a2c735c46 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/fc635e1c37354dad89e73e8a2c735c46 2023-02-08 08:55:50,219 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,220 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2 2023-02-08 08:55:50,220 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2 2023-02-08 08:55:50,221 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4978a059e5d05283e1c29386375100b5 columnFamilyName cf2 2023-02-08 08:55:50,223 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46131037d0eaa12d89cacb90c1b92e8c/cf2 of 46131037d0eaa12d89cacb90c1b92e8c into fc635e1c37354dad89e73e8a2c735c46(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:50,223 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46131037d0eaa12d89cacb90c1b92e8c: 2023-02-08 08:55:50,223 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c., storeName=46131037d0eaa12d89cacb90c1b92e8c/cf2, priority=9, startTime=1675846550165; duration=0sec 2023-02-08 08:55:50,223 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,236 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-bottom 2023-02-08 08:55:50,239 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-bottom 2023-02-08 08:55:50,242 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-bottom 2023-02-08 08:55:50,245 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-bottom 2023-02-08 08:55:50,249 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-bottom 2023-02-08 08:55:50,253 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-bottom 2023-02-08 08:55:50,255 DEBUG [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(539): loaded hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-bottom 2023-02-08 08:55:50,256 INFO [StoreOpener-4978a059e5d05283e1c29386375100b5-1] regionserver.HStore(310): Store=4978a059e5d05283e1c29386375100b5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:50,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:50,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4978a059e5d05283e1c29386375100b5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74761197, jitterRate=0.11402864754199982}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:50,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4978a059e5d05283e1c29386375100b5: Running coprocessor pre-open hook at 1675846550168Writing region info on filesystem at 1675846550168Initializing all the Stores at 1675846550172 (+4 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846550173 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846550173Cleaning up temporary data from old regions at 1675846550257 (+84 ms)Cleaning up detritus from prior splits at 1675846550258 (+1 ms)Running coprocessor post-open hooks at 1675846550259 (+1 ms)Region opened successfully at 1675846550259 2023-02-08 08:55:50,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5., pid=150, masterSystemTime=1675846550098 2023-02-08 08:55:50,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,261 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:50,261 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 08:55:50,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:50,261 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,261 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,261 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1912): 4978a059e5d05283e1c29386375100b5/cf2 is initiating minor compaction (all files) 2023-02-08 08:55:50,262 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1912): 4978a059e5d05283e1c29386375100b5/cf1 is initiating minor compaction (all files) 2023-02-08 08:55:50,262 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4978a059e5d05283e1c29386375100b5/cf2 in testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,262 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4978a059e5d05283e1c29386375100b5/cf1 in testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,262 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/.tmp, totalSize=34.9 K 2023-02-08 08:55:50,262 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-bottom] into tmpdir=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/.tmp, totalSize=34.9 K 2023-02-08 08:55:50,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=4978a059e5d05283e1c29386375100b5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:50,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550263"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846550263"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846550263"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846550263"}]},"ts":"1675846550263"} 2023-02-08 08:55:50,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-08 08:55:50,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 4978a059e5d05283e1c29386375100b5, server=jenkins-hbase4.apache.org,44651,1675846441425 in 315 msec 2023-02-08 08:55:50,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-08 08:55:50,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, ASSIGN in 475 msec 2023-02-08 08:55:50,268 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846547585 2023-02-08 08:55:50,269 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846547749 2023-02-08 08:55:50,269 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675846547585 2023-02-08 08:55:50,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=30132006af7f58dd501907fc8f96d2f7, daughterA=4978a059e5d05283e1c29386375100b5, daughterB=46131037d0eaa12d89cacb90c1b92e8c in 1.2580 sec 2023-02-08 08:55:50,269 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846548316 2023-02-08 08:55:50,270 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675846547749 2023-02-08 08:55:50,270 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846548487 2023-02-08 08:55:50,270 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675846548316 2023-02-08 08:55:50,270 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846548662 2023-02-08 08:55:50,270 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675846548487 2023-02-08 08:55:50,271 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting 896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846548833 2023-02-08 08:55:50,271 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting 927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675846548662 2023-02-08 08:55:50,271 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] compactions.Compactor(207): Compacting ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846549002 2023-02-08 08:55:50,272 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675846548833 2023-02-08 08:55:50,272 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] compactions.Compactor(207): Compacting ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675846549002 2023-02-08 08:55:50,283 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] throttle.PressureAwareThroughputController(145): 4978a059e5d05283e1c29386375100b5#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:50,288 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4978a059e5d05283e1c29386375100b5#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:50,319 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/.tmp/cf2/17b8bca1986649ef8c35f8d70b1fff26 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/17b8bca1986649ef8c35f8d70b1fff26 2023-02-08 08:55:50,324 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4978a059e5d05283e1c29386375100b5/cf2 of 4978a059e5d05283e1c29386375100b5 into 17b8bca1986649ef8c35f8d70b1fff26(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:50,324 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4978a059e5d05283e1c29386375100b5: 2023-02-08 08:55:50,324 INFO [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5., storeName=4978a059e5d05283e1c29386375100b5/cf2, priority=9, startTime=1675846550261; duration=0sec 2023-02-08 08:55:50,324 DEBUG [RS:0;jenkins-hbase4:44651-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44651] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:49624 deadline: 1675846610512, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7. is not online on jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:50,621 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:55:50,628 INFO [Listener at localhost/33105] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 08:55:50,629 DEBUG [Listener at localhost/33105] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 08:55:50,642 WARN [Listener at localhost/33105] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 08:55:50,642 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,642 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,644 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,644 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,644 INFO [Listener at localhost/33105] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-08 08:55:50,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-08 08:55:50,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,647 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,647 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 08:55:50,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846550647"}]},"ts":"1675846550647"} 2023-02-08 08:55:50,648 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-08 08:55:50,649 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-08 08:55:50,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, UNASSIGN}] 2023-02-08 08:55:50,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, UNASSIGN 2023-02-08 08:55:50,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=46131037d0eaa12d89cacb90c1b92e8c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:50,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846550651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846550651"}]},"ts":"1675846550651"} 2023-02-08 08:55:50,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 46131037d0eaa12d89cacb90c1b92e8c, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:50,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, UNASSIGN 2023-02-08 08:55:50,654 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=4978a059e5d05283e1c29386375100b5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:50,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550654"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846550654"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846550654"}]},"ts":"1675846550654"} 2023-02-08 08:55:50,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 4978a059e5d05283e1c29386375100b5, server=jenkins-hbase4.apache.org,44651,1675846441425}] 2023-02-08 08:55:50,733 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/.tmp/cf1/4fc8e290090545578814d079fe2bea25 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4fc8e290090545578814d079fe2bea25 2023-02-08 08:55:50,737 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4978a059e5d05283e1c29386375100b5/cf1 of 4978a059e5d05283e1c29386375100b5 into 4fc8e290090545578814d079fe2bea25(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:50,737 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4978a059e5d05283e1c29386375100b5: 2023-02-08 08:55:50,737 INFO [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5., storeName=4978a059e5d05283e1c29386375100b5/cf1, priority=9, startTime=1675846550260; duration=0sec 2023-02-08 08:55:50,737 DEBUG [RS:0;jenkins-hbase4:44651-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:50,747 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,748 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 08:55:50,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 46131037d0eaa12d89cacb90c1b92e8c, disabling compactions & flushes 2023-02-08 08:55:50,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. after waiting 0 ms 2023-02-08 08:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,804 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-top] to archive 2023-02-08 08:55:50,805 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:50,807 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,808 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,809 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,810 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,811 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,812 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,813 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,814 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-top, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-top] to archive 2023-02-08 08:55:50,815 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:50,816 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,817 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,817 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,818 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,819 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,820 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,821 DEBUG [StoreCloser-testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:55:50,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:50,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c. 2023-02-08 08:55:50,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 46131037d0eaa12d89cacb90c1b92e8c: Waiting for close lock at 1675846550804Running coprocessor pre-close hooks at 1675846550804Disabling compacts and flushes for region at 1675846550804Disabling writes for close at 1675846550804Writing region close event to WAL at 1675846550822 (+18 ms)Running coprocessor post-close hooks at 1675846550824 (+2 ms)Closed at 1675846550824 2023-02-08 08:55:50,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4978a059e5d05283e1c29386375100b5, disabling compactions & flushes 2023-02-08 08:55:50,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. after waiting 0 ms 2023-02-08 08:55:50,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=46131037d0eaa12d89cacb90c1b92e8c, regionState=CLOSED 2023-02-08 08:55:50,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550826"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846550826"}]},"ts":"1675846550826"} 2023-02-08 08:55:50,827 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36-bottom] to archive 2023-02-08 08:55:50,827 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:50,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-08 08:55:50,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 46131037d0eaa12d89cacb90c1b92e8c, server=jenkins-hbase4.apache.org,44651,1675846441425 in 175 msec 2023-02-08 08:55:50,829 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/bb4da817ed814fe288b4e4f770d44116.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=46131037d0eaa12d89cacb90c1b92e8c, UNASSIGN in 179 msec 2023-02-08 08:55:50,830 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4d00b2fa5a0a404f859579dcc305b75c.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,831 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/42a0b2719f7a4f969ebe1fe4403588c7.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,832 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/371a56ada5964654930bef60194dd292.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,833 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/927e2f66abc64f339e7a471b509c8895.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,833 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/a3b28368bd994b05b5d7c13f16bdec77.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,834 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/ff22f0949ef046f29aa73d320be04e36.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,836 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c-bottom, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7->hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354-bottom] to archive 2023-02-08 08:55:50,836 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:50,837 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/0fc62344523b4c5796d037078428058e.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,838 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ca7f56b3389a4a15ba54ba30e90143a0.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,839 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/41f2f112068b48b2b678f5b7cd11e1f6.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,840 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/d6e8ea64755d4d7bbc1d63643878c47a.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,841 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/3aca5001c3394b17bd930d7c618a992d.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,841 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/896e3fb926c94ef6b8fa1bd9650d167c.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,842 DEBUG [StoreCloser-testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/ccf310d74d4d462c9bd662eaf7da7354.30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 08:55:50,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:50,845 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5. 2023-02-08 08:55:50,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4978a059e5d05283e1c29386375100b5: Waiting for close lock at 1675846550826Running coprocessor pre-close hooks at 1675846550826Disabling compacts and flushes for region at 1675846550826Disabling writes for close at 1675846550826Writing region close event to WAL at 1675846550843 (+17 ms)Running coprocessor post-close hooks at 1675846550845 (+2 ms)Closed at 1675846550845 2023-02-08 08:55:50,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,847 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=4978a059e5d05283e1c29386375100b5, regionState=CLOSED 2023-02-08 08:55:50,847 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846550847"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846550847"}]},"ts":"1675846550847"} 2023-02-08 08:55:50,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-08 08:55:50,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 4978a059e5d05283e1c29386375100b5, server=jenkins-hbase4.apache.org,44651,1675846441425 in 193 msec 2023-02-08 08:55:50,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-08 08:55:50,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=4978a059e5d05283e1c29386375100b5, UNASSIGN in 199 msec 2023-02-08 08:55:50,851 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846550851"}]},"ts":"1675846550851"} 2023-02-08 08:55:50,852 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-08 08:55:50,854 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-08 08:55:50,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 211 msec 2023-02-08 08:55:50,948 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,949 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 08:55:50,950 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-08 08:55:50,950 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,950 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-08 08:55:50,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,952 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,953 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:50,953 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:50,953 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 08:55:50,956 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,957 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,957 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,958 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/recovered.edits] 2023-02-08 08:55:50,958 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/recovered.edits] 2023-02-08 08:55:50,958 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2, FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/recovered.edits] 2023-02-08 08:55:50,961 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/358d69053326413bace24489487bb617 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf1/358d69053326413bace24489487bb617 2023-02-08 08:55:50,961 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4fc8e290090545578814d079fe2bea25 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf1/4fc8e290090545578814d079fe2bea25 2023-02-08 08:55:50,962 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/371a56ada5964654930bef60194dd292 2023-02-08 08:55:50,963 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/fc635e1c37354dad89e73e8a2c735c46 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/cf2/fc635e1c37354dad89e73e8a2c735c46 2023-02-08 08:55:50,963 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/42a0b2719f7a4f969ebe1fe4403588c7 2023-02-08 08:55:50,963 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/17b8bca1986649ef8c35f8d70b1fff26 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/cf2/17b8bca1986649ef8c35f8d70b1fff26 2023-02-08 08:55:50,964 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/4d00b2fa5a0a404f859579dcc305b75c 2023-02-08 08:55:50,965 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/927e2f66abc64f339e7a471b509c8895 2023-02-08 08:55:50,965 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c/recovered.edits/90.seqid 2023-02-08 08:55:50,966 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/recovered.edits/90.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5/recovered.edits/90.seqid 2023-02-08 08:55:50,966 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/46131037d0eaa12d89cacb90c1b92e8c 2023-02-08 08:55:50,966 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/4978a059e5d05283e1c29386375100b5 2023-02-08 08:55:50,966 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/a3b28368bd994b05b5d7c13f16bdec77 2023-02-08 08:55:50,967 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/bb4da817ed814fe288b4e4f770d44116 2023-02-08 08:55:50,968 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf1/ff22f0949ef046f29aa73d320be04e36 2023-02-08 08:55:50,970 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/0fc62344523b4c5796d037078428058e 2023-02-08 08:55:50,971 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/3aca5001c3394b17bd930d7c618a992d 2023-02-08 08:55:50,972 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/41f2f112068b48b2b678f5b7cd11e1f6 2023-02-08 08:55:50,973 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/896e3fb926c94ef6b8fa1bd9650d167c 2023-02-08 08:55:50,974 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ca7f56b3389a4a15ba54ba30e90143a0 2023-02-08 08:55:50,975 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354 to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/ccf310d74d4d462c9bd662eaf7da7354 2023-02-08 08:55:50,976 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/cf2/d6e8ea64755d4d7bbc1d63643878c47a 2023-02-08 08:55:50,978 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/recovered.edits/85.seqid to hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/archive/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7/recovered.edits/85.seqid 2023-02-08 08:55:50,978 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/.tmp/data/default/testSplitTableRegion/30132006af7f58dd501907fc8f96d2f7 2023-02-08 08:55:50,978 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 08:55:50,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,981 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-08 08:55:50,982 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-08 08:55:50,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-08 08:55:50,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846550983"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:50,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846550983"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:50,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846550983"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:50,984 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:55:50,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 30132006af7f58dd501907fc8f96d2f7, NAME => 'testSplitTableRegion,,1675846547082.30132006af7f58dd501907fc8f96d2f7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4978a059e5d05283e1c29386375100b5, NAME => 'testSplitTableRegion,,1675846549010.4978a059e5d05283e1c29386375100b5.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 46131037d0eaa12d89cacb90c1b92e8c, NAME => 'testSplitTableRegion,41,1675846549010.46131037d0eaa12d89cacb90c1b92e8c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 08:55:50,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-08 08:55:50,984 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846550984"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:50,985 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-08 08:55:50,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 08:55:50,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 37 msec 2023-02-08 08:55:51,053 INFO [Listener at localhost/33105] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:51,054 INFO [Listener at localhost/33105] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:51,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33127] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 08:55:51,054 INFO [Listener at localhost/33105] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-08 08:55:51,068 INFO [Listener at localhost/33105] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=372 (was 371) - Thread LEAK? -, OpenFileDescriptor=838 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=120 (was 130), ProcessCount=168 (was 168), AvailableMemoryMB=3946 (was 3973) 2023-02-08 08:55:51,068 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-08 08:55:51,068 INFO [Listener at localhost/33105] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:51,069 DEBUG [Listener at localhost/33105] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58b3884f to 127.0.0.1:53039 2023-02-08 08:55:51,069 DEBUG [Listener at localhost/33105] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,069 DEBUG [Listener at localhost/33105] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-08 08:55:51,070 DEBUG [Listener at localhost/33105] util.JVMClusterUtil(257): Found active master hash=1531146278, stopped=false 2023-02-08 08:55:51,070 INFO [Listener at localhost/33105] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:55:51,071 INFO [Listener at localhost/33105] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:55:51,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:55:51,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:55:51,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:55:51,071 DEBUG [Listener at localhost/33105] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22458b66 to 127.0.0.1:53039 2023-02-08 08:55:51,072 DEBUG [Listener at localhost/33105] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,44651,1675846441425' ***** 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36805,1675846441472' ***** 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39055,1675846441504' ***** 2023-02-08 08:55:51,072 INFO [Listener at localhost/33105] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 08:55:51,073 INFO [RS:0;jenkins-hbase4:44651] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 08:55:51,073 INFO [RS:2;jenkins-hbase4:39055] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 08:55:51,073 INFO [RS:1;jenkins-hbase4:36805] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 08:55:51,073 INFO [RS:1;jenkins-hbase4:36805] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 08:55:51,073 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 08:55:51,073 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 08:55:51,073 INFO [RS:1;jenkins-hbase4:36805] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 08:55:51,073 INFO [RS:2;jenkins-hbase4:39055] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 08:55:51,073 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:51,073 INFO [RS:2;jenkins-hbase4:39055] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 08:55:51,073 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 08:55:51,073 DEBUG [RS:1;jenkins-hbase4:36805] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2dd3b496 to 127.0.0.1:53039 2023-02-08 08:55:51,073 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(3304): Received CLOSE for 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:55:51,073 DEBUG [RS:1;jenkins-hbase4:36805] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,074 INFO [RS:1;jenkins-hbase4:36805] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 08:55:51,074 INFO [RS:0;jenkins-hbase4:44651] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 08:55:51,074 INFO [RS:0;jenkins-hbase4:44651] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 08:55:51,074 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:51,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a7e5b1dad112219d73dc56e1f1e8ebf, disabling compactions & flushes 2023-02-08 08:55:51,074 INFO [RS:1;jenkins-hbase4:36805] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 08:55:51,074 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:55:51,074 DEBUG [RS:0;jenkins-hbase4:44651] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3cf43ab3 to 127.0.0.1:53039 2023-02-08 08:55:51,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:55:51,074 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:51,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. after waiting 0 ms 2023-02-08 08:55:51,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:55:51,074 DEBUG [RS:0;jenkins-hbase4:44651] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,074 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 9a7e5b1dad112219d73dc56e1f1e8ebf 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-08 08:55:51,074 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,44651,1675846441425; all regions closed. 2023-02-08 08:55:51,074 INFO [RS:1;jenkins-hbase4:36805] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 08:55:51,074 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-08 08:55:51,074 DEBUG [RS:2;jenkins-hbase4:39055] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0ca17ad2 to 127.0.0.1:53039 2023-02-08 08:55:51,074 DEBUG [RS:2;jenkins-hbase4:39055] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,074 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-08 08:55:51,074 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1479): Online Regions={9a7e5b1dad112219d73dc56e1f1e8ebf=hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf.} 2023-02-08 08:55:51,074 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-08 08:55:51,075 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-08 08:55:51,075 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 08:55:51,075 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 08:55:51,075 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 08:55:51,075 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 08:55:51,075 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 08:55:51,075 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-08 08:55:51,075 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-08 08:55:51,075 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1505): Waiting on 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:55:51,085 DEBUG [RS:0;jenkins-hbase4:44651] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs 2023-02-08 08:55:51,085 INFO [RS:0;jenkins-hbase4:44651] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44651%2C1675846441425:(num 1675846443073) 2023-02-08 08:55:51,085 DEBUG [RS:0;jenkins-hbase4:44651] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,085 INFO [RS:0;jenkins-hbase4:44651] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,090 INFO [RS:0;jenkins-hbase4:44651] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 08:55:51,090 INFO [RS:0;jenkins-hbase4:44651] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 08:55:51,090 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:55:51,090 INFO [RS:0;jenkins-hbase4:44651] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 08:55:51,090 INFO [RS:0;jenkins-hbase4:44651] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 08:55:51,092 INFO [RS:0;jenkins-hbase4:44651] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44651 2023-02-08 08:55:51,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/.tmp/info/5092f403e0e24ac3b70e36936a2dabdf 2023-02-08 08:55:51,264 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,264 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,264 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,267 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5c263ed8 rejected from java.util.concurrent.ThreadPoolExecutor@5819a184[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44651,1675846441425 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,267 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,267 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@41f1fbde rejected from java.util.concurrent.ThreadPoolExecutor@5819a184[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,268 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/info/980a406a1c05484e8db17ce89c45e7b3 2023-02-08 08:55:51,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/.tmp/info/5092f403e0e24ac3b70e36936a2dabdf as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/info/5092f403e0e24ac3b70e36936a2dabdf 2023-02-08 08:55:51,271 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44651,1675846441425] 2023-02-08 08:55:51,271 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44651,1675846441425; numProcessing=1 2023-02-08 08:55:51,271 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 980a406a1c05484e8db17ce89c45e7b3 2023-02-08 08:55:51,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/info/5092f403e0e24ac3b70e36936a2dabdf, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 08:55:51,275 DEBUG [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-08 08:55:51,275 DEBUG [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1505): Waiting on 9a7e5b1dad112219d73dc56e1f1e8ebf 2023-02-08 08:55:51,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 9a7e5b1dad112219d73dc56e1f1e8ebf in 202ms, sequenceid=6, compaction requested=false 2023-02-08 08:55:51,294 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/rep_barrier/af4b66a67e9c4156ae4fe7ce1972024e 2023-02-08 08:55:51,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/namespace/9a7e5b1dad112219d73dc56e1f1e8ebf/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-08 08:55:51,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:51,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:55:51,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a7e5b1dad112219d73dc56e1f1e8ebf: Waiting for close lock at 1675846551074Running coprocessor pre-close hooks at 1675846551074Disabling compacts and flushes for region at 1675846551074Disabling writes for close at 1675846551074Obtaining lock to block concurrent updates at 1675846551074Preparing flush snapshotting stores in 9a7e5b1dad112219d73dc56e1f1e8ebf at 1675846551074Finished memstore snapshotting hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675846551074Flushing stores of hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. at 1675846551075 (+1 ms)Flushing 9a7e5b1dad112219d73dc56e1f1e8ebf/info: creating writer at 1675846551075Flushing 9a7e5b1dad112219d73dc56e1f1e8ebf/info: appending metadata at 1675846551082 (+7 ms)Flushing 9a7e5b1dad112219d73dc56e1f1e8ebf/info: closing flushed file at 1675846551082Flushing 9a7e5b1dad112219d73dc56e1f1e8ebf/info: reopening flushed file at 1675846551271 (+189 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 9a7e5b1dad112219d73dc56e1f1e8ebf in 202ms, sequenceid=6, compaction requested=false at 1675846551276 (+5 ms)Writing region close event to WAL at 1675846551284 (+8 ms)Running coprocessor post-close hooks at 1675846551295 (+11 ms)Closed at 1675846551296 (+1 ms) 2023-02-08 08:55:51,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675846443603.9a7e5b1dad112219d73dc56e1f1e8ebf. 2023-02-08 08:55:51,298 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af4b66a67e9c4156ae4fe7ce1972024e 2023-02-08 08:55:51,306 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/table/23ee302060704947ab43cb9ea0bf7587 2023-02-08 08:55:51,309 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 23ee302060704947ab43cb9ea0bf7587 2023-02-08 08:55:51,309 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/info/980a406a1c05484e8db17ce89c45e7b3 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/info/980a406a1c05484e8db17ce89c45e7b3 2023-02-08 08:55:51,312 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 980a406a1c05484e8db17ce89c45e7b3 2023-02-08 08:55:51,312 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/info/980a406a1c05484e8db17ce89c45e7b3, entries=59, sequenceid=210, filesize=11.1 K 2023-02-08 08:55:51,313 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/rep_barrier/af4b66a67e9c4156ae4fe7ce1972024e as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/rep_barrier/af4b66a67e9c4156ae4fe7ce1972024e 2023-02-08 08:55:51,315 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af4b66a67e9c4156ae4fe7ce1972024e 2023-02-08 08:55:51,315 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/rep_barrier/af4b66a67e9c4156ae4fe7ce1972024e, entries=24, sequenceid=210, filesize=7.5 K 2023-02-08 08:55:51,316 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/.tmp/table/23ee302060704947ab43cb9ea0bf7587 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/table/23ee302060704947ab43cb9ea0bf7587 2023-02-08 08:55:51,318 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 23ee302060704947ab43cb9ea0bf7587 2023-02-08 08:55:51,318 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/table/23ee302060704947ab43cb9ea0bf7587, entries=36, sequenceid=210, filesize=8.0 K 2023-02-08 08:55:51,319 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 244ms, sequenceid=210, compaction requested=false 2023-02-08 08:55:51,323 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-08 08:55:51,324 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 08:55:51,324 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 08:55:51,324 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 08:55:51,324 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675846551075Running coprocessor pre-close hooks at 1675846551075Disabling compacts and flushes for region at 1675846551075Disabling writes for close at 1675846551075Obtaining lock to block concurrent updates at 1675846551075Preparing flush snapshotting stores in 1588230740 at 1675846551075Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675846551075Flushing stores of hbase:meta,,1.1588230740 at 1675846551076 (+1 ms)Flushing 1588230740/info: creating writer at 1675846551076Flushing 1588230740/info: appending metadata at 1675846551086 (+10 ms)Flushing 1588230740/info: closing flushed file at 1675846551086Flushing 1588230740/rep_barrier: creating writer at 1675846551271 (+185 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675846551274 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675846551274Flushing 1588230740/table: creating writer at 1675846551298 (+24 ms)Flushing 1588230740/table: appending metadata at 1675846551301 (+3 ms)Flushing 1588230740/table: closing flushed file at 1675846551301Flushing 1588230740/info: reopening flushed file at 1675846551310 (+9 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675846551313 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1675846551316 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 244ms, sequenceid=210, compaction requested=false at 1675846551319 (+3 ms)Writing region close event to WAL at 1675846551321 (+2 ms)Running coprocessor post-close hooks at 1675846551323 (+2 ms)Closed at 1675846551324 (+1 ms) 2023-02-08 08:55:51,325 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-08 08:55:51,371 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,371 INFO [RS:0;jenkins-hbase4:44651] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,44651,1675846441425; zookeeper connection closed. 2023-02-08 08:55:51,371 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@39cc5c5a rejected from java.util.concurrent.ThreadPoolExecutor@5819a184[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,371 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:44651-0x10033233bf30001, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,372 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2fc7df58] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2fc7df58 2023-02-08 08:55:51,372 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3355e0df rejected from java.util.concurrent.ThreadPoolExecutor@5819a184[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,372 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,44651,1675846441425 already deleted, retry=false 2023-02-08 08:55:51,372 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44651,1675846441425 expired; onlineServers=2 2023-02-08 08:55:51,473 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:51,476 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36805,1675846441472; all regions closed. 2023-02-08 08:55:51,476 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39055,1675846441504; all regions closed. 2023-02-08 08:55:51,482 DEBUG [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs 2023-02-08 08:55:51,482 INFO [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36805%2C1675846441472.meta:.meta(num 1675846443330) 2023-02-08 08:55:51,483 DEBUG [RS:2;jenkins-hbase4:39055] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs 2023-02-08 08:55:51,483 INFO [RS:2;jenkins-hbase4:39055] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39055%2C1675846441504:(num 1675846443073) 2023-02-08 08:55:51,483 DEBUG [RS:2;jenkins-hbase4:39055] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,483 INFO [RS:2;jenkins-hbase4:39055] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,483 INFO [RS:2;jenkins-hbase4:39055] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 08:55:51,484 INFO [RS:2;jenkins-hbase4:39055] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 08:55:51,484 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:55:51,484 INFO [RS:2;jenkins-hbase4:39055] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 08:55:51,484 INFO [RS:2;jenkins-hbase4:39055] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 08:55:51,485 INFO [RS:2;jenkins-hbase4:39055] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39055 2023-02-08 08:55:51,487 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/WALs/jenkins-hbase4.apache.org,36805,1675846441472/jenkins-hbase4.apache.org%2C36805%2C1675846441472.1675846443073 not finished, retry = 0 2023-02-08 08:55:51,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:51,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,487 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39055,1675846441504 2023-02-08 08:55:51,488 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6440c92b rejected from java.util.concurrent.ThreadPoolExecutor@607f517b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,488 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39055,1675846441504] 2023-02-08 08:55:51,488 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39055,1675846441504; numProcessing=2 2023-02-08 08:55:51,490 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39055,1675846441504 already deleted, retry=false 2023-02-08 08:55:51,490 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39055,1675846441504 expired; onlineServers=1 2023-02-08 08:55:51,589 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,589 INFO [RS:2;jenkins-hbase4:39055] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39055,1675846441504; zookeeper connection closed. 2023-02-08 08:55:51,589 DEBUG [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/oldWALs 2023-02-08 08:55:51,589 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7679e2d6 rejected from java.util.concurrent.ThreadPoolExecutor@607f517b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,589 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@33609ecc] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@33609ecc 2023-02-08 08:55:51,589 INFO [RS:1;jenkins-hbase4:36805] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36805%2C1675846441472:(num 1675846443073) 2023-02-08 08:55:51,589 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:39055-0x10033233bf30003, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,589 DEBUG [RS:1;jenkins-hbase4:36805] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,589 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@e7c1bc3 rejected from java.util.concurrent.ThreadPoolExecutor@607f517b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,589 INFO [RS:1;jenkins-hbase4:36805] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:55:51,590 INFO [RS:1;jenkins-hbase4:36805] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-08 08:55:51,590 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:55:51,591 INFO [RS:1;jenkins-hbase4:36805] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36805 2023-02-08 08:55:51,592 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36805,1675846441472 2023-02-08 08:55:51,592 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:55:51,592 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a5cb826 rejected from java.util.concurrent.ThreadPoolExecutor@6e6321b5[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,593 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36805,1675846441472] 2023-02-08 08:55:51,593 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36805,1675846441472; numProcessing=3 2023-02-08 08:55:51,594 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36805,1675846441472 already deleted, retry=false 2023-02-08 08:55:51,594 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36805,1675846441472 expired; onlineServers=0 2023-02-08 08:55:51,594 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33127,1675846440297' ***** 2023-02-08 08:55:51,594 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-08 08:55:51,594 DEBUG [M:0;jenkins-hbase4:33127] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1a7e7a1d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33127,1675846440297 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33127,1675846440297; all regions closed. 2023-02-08 08:55:51,595 DEBUG [M:0;jenkins-hbase4:33127] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,595 DEBUG [M:0;jenkins-hbase4:33127] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-08 08:55:51,595 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-08 08:55:51,595 DEBUG [M:0;jenkins-hbase4:33127] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-08 08:55:51,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846442732] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846442732,5,FailOnTimeoutGroup] 2023-02-08 08:55:51,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846442732] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846442732,5,FailOnTimeoutGroup] 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-08 08:55:51,595 DEBUG [M:0;jenkins-hbase4:33127] master.HMaster(1502): Stopping service threads 2023-02-08 08:55:51,595 INFO [M:0;jenkins-hbase4:33127] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-08 08:55:51,595 ERROR [M:0;jenkins-hbase4:33127] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-08 08:55:51,596 INFO [M:0;jenkins-hbase4:33127] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:55:51,596 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-08 08:55:51,597 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-08 08:55:51,597 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:55:51,597 DEBUG [M:0;jenkins-hbase4:33127] zookeeper.ZKUtil(398): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-08 08:55:51,597 WARN [M:0;jenkins-hbase4:33127] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-08 08:55:51,597 INFO [M:0;jenkins-hbase4:33127] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:55:51,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:55:51,597 INFO [M:0;jenkins-hbase4:33127] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-08 08:55:51,597 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 08:55:51,597 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:55:51,597 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:55:51,597 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 08:55:51,597 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:55:51,597 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.17 KB heapSize=719.09 KB 2023-02-08 08:55:51,693 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,693 INFO [RS:1;jenkins-hbase4:36805] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36805,1675846441472; zookeeper connection closed. 2023-02-08 08:55:51,693 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@151208d6 rejected from java.util.concurrent.ThreadPoolExecutor@6e6321b5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,693 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): regionserver:36805-0x10033233bf30002, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:51,693 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6d19fbb6 rejected from java.util.concurrent.ThreadPoolExecutor@6e6321b5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:51,693 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@baf2103] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@baf2103 2023-02-08 08:55:51,694 INFO [Listener at localhost/33105] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-08 08:55:52,009 INFO [M:0;jenkins-hbase4:33127] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.17 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bad5d5508ca24f6bb109c984330e8c99 2023-02-08 08:55:52,012 INFO [M:0;jenkins-hbase4:33127] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bad5d5508ca24f6bb109c984330e8c99 2023-02-08 08:55:52,012 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bad5d5508ca24f6bb109c984330e8c99 as hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bad5d5508ca24f6bb109c984330e8c99 2023-02-08 08:55:52,015 INFO [M:0;jenkins-hbase4:33127] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bad5d5508ca24f6bb109c984330e8c99 2023-02-08 08:55:52,015 INFO [M:0;jenkins-hbase4:33127] regionserver.HStore(1080): Added hdfs://localhost:42159/user/jenkins/test-data/b462f541-04cc-3f5e-8118-91e6f714d6eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bad5d5508ca24f6bb109c984330e8c99, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-08 08:55:52,016 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegion(2947): Finished flush of dataSize ~608.17 KB/622761, heapSize ~719.08 KB/736336, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 419ms, sequenceid=1282, compaction requested=false 2023-02-08 08:55:52,017 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:55:52,017 DEBUG [M:0;jenkins-hbase4:33127] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675846551597Disabling compacts and flushes for region at 1675846551597Disabling writes for close at 1675846551597Obtaining lock to block concurrent updates at 1675846551597Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675846551597Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622761, getHeapSize=736336, getOffHeapSize=0, getCellsCount=1499 at 1675846551598 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675846551598Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675846551598Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675846551601 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675846551601Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675846552013 (+412 ms)Finished flush of dataSize ~608.17 KB/622761, heapSize ~719.08 KB/736336, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 419ms, sequenceid=1282, compaction requested=false at 1675846552016 (+3 ms)Writing region close event to WAL at 1675846552017 (+1 ms)Closed at 1675846552017 2023-02-08 08:55:52,020 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:55:52,020 INFO [M:0;jenkins-hbase4:33127] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-08 08:55:52,021 INFO [M:0;jenkins-hbase4:33127] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33127 2023-02-08 08:55:52,022 DEBUG [M:0;jenkins-hbase4:33127] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33127,1675846440297 already deleted, retry=false 2023-02-08 08:55:52,124 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:52,124 INFO [M:0;jenkins-hbase4:33127] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33127,1675846440297; zookeeper connection closed. 2023-02-08 08:55:52,124 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@727a7a22 rejected from java.util.concurrent.ThreadPoolExecutor@791ec0f9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:52,124 DEBUG [Listener at localhost/33105-EventThread] zookeeper.ZKWatcher(600): master:33127-0x10033233bf30000, quorum=127.0.0.1:53039, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:55:52,124 ERROR [Listener at localhost/33105-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4c069a3c rejected from java.util.concurrent.ThreadPoolExecutor@791ec0f9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:55:52,125 WARN [Listener at localhost/33105] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 08:55:52,129 INFO [Listener at localhost/33105] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:55:52,231 WARN [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 5eb5dbca-d965-4f2a-b02d-63f5d6c9e42f) service to localhost/127.0.0.1:42159 2023-02-08 08:55:52,234 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data5/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,235 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data6/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,237 WARN [Listener at localhost/33105] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 08:55:52,240 INFO [Listener at localhost/33105] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:55:52,341 WARN [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 260f0410-1e75-4d9d-aa4e-3ba6e41b0476) service to localhost/127.0.0.1:42159 2023-02-08 08:55:52,342 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data3/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,343 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data4/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,345 WARN [Listener at localhost/33105] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 08:55:52,347 INFO [Listener at localhost/33105] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:55:52,373 WARN [BP-27682754-172.31.14.131-1675846435525 heartbeating to localhost/127.0.0.1:42159] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-27682754-172.31.14.131-1675846435525 (Datanode Uuid 39265e67-bcfa-4780-a199-0487aaa87ffd) service to localhost/127.0.0.1:42159 2023-02-08 08:55:52,374 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data1/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,374 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/82b4178c-0347-78cf-11ad-783d3d56b6b1/cluster_5613d4a7-5c37-baad-5716-d7c04318794d/dfs/data/data2/current/BP-27682754-172.31.14.131-1675846435525] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:55:52,482 INFO [Listener at localhost/33105] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:55:52,674 INFO [Listener at localhost/33105] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-08 08:55:52,919 INFO [Listener at localhost/33105] hbase.HBaseTestingUtility(1293): Minicluster is down